Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using primary or unique keys to construct kafka keys #88

Open
gerbrand opened this issue Apr 7, 2021 · 0 comments
Open

Using primary or unique keys to construct kafka keys #88

gerbrand opened this issue Apr 7, 2021 · 0 comments

Comments

@gerbrand
Copy link

gerbrand commented Apr 7, 2021

The jdbc-source-connector does not create kafka keys by default. I found some question on stackoverlow that mentions using a kafka transformer. Instead why not use primary keys as keys in Kafka? What fields are primary keys is already retrieved via GenericDatabaseDialect amd stored in ColumnDefinition. I'd think it would be not too hard to extend the jdbc-source-connecor to use the analoguous property PK_MODE. When set to RECORD_KEY the source connector could create keys as well using primary keys.
Advantage would be that compacting could be enabled for topics created using the jdbc-source-connector.

At first glance I'd think this feature wouldn't be too hard to add. But before I'd start working an a pull-request, Are there any reasons why this feature is not added? Is there additional complexity that I am missing?

@gerbrand gerbrand changed the title Using primary keys to construct kafka keys Using primary or unique keys to construct kafka keys Apr 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant