Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

java.util.ConcurrentModificationException during task rebalancing #600

Open
smj19 opened this issue Dec 14, 2021 · 1 comment
Open

java.util.ConcurrentModificationException during task rebalancing #600

smj19 opened this issue Dec 14, 2021 · 1 comment

Comments

@smj19
Copy link

smj19 commented Dec 14, 2021

We're using 5.4.3 version of Confluent and observed that during a task rebalancing, it threw the below exception for multiple partitions:

[ERROR] 2021-12-12 03:34:16,891 org.apache.kafka.connect.runtime.WorkerTask doRun - WorkerSinkTask{id=hdfs-connector-prod-399} Task threw an uncaught and unrecoverable exception
java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextNode(Unknown Source)
	at java.util.HashMap$KeyIterator.next(Unknown Source)
	at io.confluent.connect.hdfs.TopicPartitionWriter.close(TopicPartitionWriter.java:463)
	at io.confluent.connect.hdfs.DataWriter.close(DataWriter.java:457)
	at io.confluent.connect.hdfs.HdfsSinkTask.close(HdfsSinkTask.java:161)

It looks like this is thrown in this for loop
https://github.com/confluentinc/kafka-connect-hdfs/blob/5.4.3-post/src/main/java/io/confluent/connect/hdfs/TopicPartitionWriter.java#L463
It is looping over writers Map and also removing the encodedPartition from the Map here

io.confluent.connect.storage.format.RecordWriter writer = writers.remove(encodedPartition);

Could that be the reason for this exception?

@smj19
Copy link
Author

smj19 commented Dec 16, 2021

Can someone help with this, please? We're seeing this issue in Production.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant