Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incremental Co-operative Rebalancing Support for HDFS Connector #625

Open
ghost opened this issue Aug 12, 2022 · 0 comments
Open

Incremental Co-operative Rebalancing Support for HDFS Connector #625

ghost opened this issue Aug 12, 2022 · 0 comments

Comments

@ghost
Copy link

ghost commented Aug 12, 2022

Hi Team,
@ConfluentJenkins
Just wanted to know if the HDFS connector has the support for the "Incremental Rebalance". As I can see that "TopicPartitionWriters" Map is being cleared on every close call, which is correct if we consider the stop the world scenario. But in Incremental mode there can be case that only few of the partitions have been revoked and task has some partitions with it to read from, In that scenario it will be throwing NullPointerException at the following part of the code.

topicPartitionWriters.get(tp).buffer(record);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants