i writing trident topology process stream of data kafka , feed in redis , cassandra. able write data in cassandra. write same data in redis.
is there way duplicate tuples , branch in 2 flow 1 goes in redis , goes in cassandra?
for trident can go smth this:
tridenttopology topology = new tridenttopology(); stream stream = topology.newstream("myspout", spout); stream.partitionpersist(...); // redis stream.partitionpersist(...); // cassandra so saving data stream both databases in parallel.
however think if such parallel thing should done inside single topology or if having 2 different topologies reading same topic better idea. imagine cassandra cluster goes down. in case of 2 topologies you'll still able continue saving data redis. if there's single topology, every tuple failed go cassandra result in failedexception trigger replaying , every subsequent replay of tuple involve saving tuple redis once again unnecessarily.
Comments
Post a Comment