- Share a single kafka server across multiple CDAP instance.
This can be achieved by using the same zookeeper quorum to be used for all clients that consumes kafka. Currently the zookeeper quorum property that is used by kafka clients is also being used by system services for service registration and discovery.
The change that would be needed is to add a separate property that can be used only for kafka consumers ( kafka.zookeeper.quorum). Update the client code to use this property instead of the generic zookeeper.quorum property. With this change and the kafka topic prefixes a single kafka server can be re-used across multiple instances of CDAP.