Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out_rdkafka2: enable idempotency for kafka producers #501

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dipendra-singh
Copy link

Which issue(s) this PR fixes:
None.

What this PR does / why we need it:
In order to enable Idempotency for kafka producers, rdkafka allowes us to enable this. (here)

Docs Changes:
README.md : Added to usage template for better visibility.

Release Note:
Same as the title.

Signed-off-by: Dipendra Singh <dipendra.singh@rubrik.com>
@ashie
Copy link
Member

ashie commented Dec 6, 2023

Any rdkafka options can be set by rdkafka_options parameter.
e.g.)

<match tag>
  @type rdkafka2
   rdkafka_options {                                                                                                                                        
     "enable.idempotence" : true
   }
   ....
</match>

Is there any reason to add a new specific parameter for idempotence?

@dipendra-singh
Copy link
Author

@ashie To keep the consistency between this and other plugins. One needs to explore rdkafka options to have similar features to that of other plugins.

@ashie
Copy link
Member

ashie commented Dec 6, 2023

Ah, I see. Thanks for clarification.
It seems reasonable.

config_param :idempotent, :bool, :default => false, :desc => 'Enable idempotent producer'

@dipendra-singh
Copy link
Author

hey @ashie
Can we close this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants