scala - How to build efficient Kafka broker healthcheck? -

in app perform kind of health check of kafka cluster.

currently make topicmetadatarequest detect dead brokers:

  future {     // fail if kafka unavailable     consumer.send(new topicmetadatarequest(seq("health-check-topic"), 1))   } 

unfortunately call produces huge network traffic, because of cluster topology/settings.

is there better way check kafka brokers? need simple true/false indicator.

i recommend use yahoo kafka manager, provides information related kafka setup. (e.g. bytes sent/consumed on time interval). tool can used managing kafka cluster.

it exposes restful api , can consume these api in own application, if needed. follow following link access it.

https://github.com/yahoo/kafka-manager