Configure Kafka broker to use Zookeeper chroot path

[a-narenji]

We need to configure Kafka brokers to use Zookeeper chroot path. By default, Kafka creates znodes in root of Zookeeper.
In order to separate app and Kafka configurations, it is necessary to set different chroot path for Kafka.

Is it possible to have Kafka with chroot Zookeeper path provisioned by cp-ansible?

As an example. we need cp-ansible to configure zookeeper.connect to have a value like: hostname1:port1,hostname2:port2,hostname3:port3/chroot/path.

[JumaX]

@a-narenji Are you using an existing zookeeper cluster for multiple applications? If so, we generally do not recommend this, we recommend you let cp-ansible install zookeeper and manage it. We cannot guarantee that whichever version you may already be running is compatible etc.

You can override all configurations via custom properties as per our documentation here:

https://docs.confluent.io/ansible/current/ansible-configure.html#set-custom-component-properties

So you could set the properties to your desired configuration on the Broker, however if we are not managing zookeeper for you, you would need to configure that yourself manually.

[a-narenji]

@JumaX Many thanks for your reply.
We are not using existing Zookeeper cluster, and we use the same Zookeeper which cp-ansible set it up for us.
Our develop team may need to use Zookeeper for another purpose, and it may not be reasonable to have a second Zookeeper cluster dedicated to develop team requirements.

So, it would be helpful that cp-ansible let us to have chroot path in zookeeper.connect property.
Additionally, since we need to enable SASL, it is not enough to change zookeeper.connect only, but also, it is required to change Create SCRAM Users and Create SCRAM 256 Users tasks in such a way that cp-ansible be able to create SCRAM users to use Zookeeper chroot path.

[JumaX]

@a-narenji Thanks for the additional details. Unfortunately, at this time we do not support this configuration. Specifically around the scram users. Of additional note, is that Zookeeper is being removed as a kafka dependency soon, so we will review this and see if it makes sense to allow this modification with zookeeper for a future release.

[domenicbove]

@a-narenji Can you please review: #694

[a-narenji]

@domenicbove We checked it in our test environment.
It worked great.
Many thanks for adding this feature.