Upgrade Kafka version to 3.4 #1973
Conversation
|
@efeg, @CCisGG, @viktorsomogyi |
|
Hi @egyedt , thank you for taking care of this! The reason we keep kafka-version at 3.1* is LinkedIn is still using kafka 3.0 ~ 3.1. We anticipate we'll stay on that version for at least another few months. Please feel free to update the kafka dependency version on your fork of CC, but we'll keep the version as what it is for now in open source CC. |
|
@CCisGG Can we upgrade the Kafka client used by cruise-control now? Or do we need to fork the project? I would expect the clients to be backward compatible. Forward compatible can be a bit more tricky. We want to connect to our clusters using PEM certificates avoiding the pesky keystores (with passwords). That requires a newer client version. |
Hi @erikgb , thanks for bringing this up. Could you please elaborate a bit more on the "We want to connect to our clusters using PEM certificates avoiding the pesky keystores (with passwords). That requires a newer client version."? Do you mean that using current CC code cannot connect your CC instance to your kafka cluster? |
|
@CCisGG We run CC on Kubernetes, and have automation to issue certificates used to authenticate to our external Kafka clusters. This automation is migrating away from legacy keystores/truststores (both JKS and PKCS#12) to pure PEM format with unencrypted private keys. The current Kafka client used in CC does not allow for this, as it doesn't include https://issues.apache.org/jira/browse/KAFKA-12703. To use unencrypted private keys with PEM, the Kafka client version must be at least 3.3.0. |
This PR resolves #1960.