Violating partition assignment

[ksdvishnukumar]

Description

I came across the below scenario in one of environment. I have a Topic (EventHub) with 3 partitions. I spanned up 3 consumers ideally each consumer should be assigned with one partition each. But from the PartitionAssignmenedHandler I see that, one consumer(Consumer1) took all the 3 partitions. But I observed that, Consumer2 received the partition 1 data even though there is no assignment to Consumer2 (Via PartitionAssignmenedHandler) and No revoke/loss assignment to Consumer1 (SetPartitionsRevokedHandler/SetPartitionsLostHandler). The core of the partition distribution itself violating which leads to duplicate message receiving. I understand that there is no rebalancing happened in this case and also no error/exception occurred.

Consumers are deployed to AKS and consumer are connecting to Azure Eventhub using Confluent Kafka Package (2.1.1)

How to reproduce

Could not be able to reproduce in the lower environment as this issue happened in production so there is no way to get the Debug logs.

Checklist

Please provide the following information:

• A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
  There is no specific code could reproduce it

• Confluent.Kafka nuget version. (2.1.1)

• Apache Kafka version. - Using Azure Eventhub

• Client configuration.
  AutoOffsetReset=Earliest
  EnableAutoOffsetStore = true
  EnableAutoCommit = false
  AutoCommitIntervalMs = 10Seconds
  EnablePartitionEof = true
  ConnectionsMaxIdleMs = 3 Min
  PartitionAssignmentStrategy = PartitionAssignmentStrategy.CooperativeSticky
  IsolationLevel = IsolationLevel.ReadUncommitted
  SocketNagleDisable = true
  SocketKeepaliveEnable = true
  MetadataMaxAgeMs = 180000
  SessionTimeoutMs = 30000
  MaxPollIntervalMs = 300000
  CancellationDelayMaxMs = 200,

• Operating system. Ubuntu

• Provide logs (with "debug" : "..." as necessary in configuration). As its happened in production no way to enable debug as it generates plenty of logs and also not able to reproduce it in lower environment

• Provide broker log log-excerpts.csv.

• Critical issue. Yes

[ksdvishnukumar]

Hi @mhowlett , @edenhill

Is there any workaround on this can be implemented based on the Consumer configuration?