Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance Kafka consumer performance #10989

Open
wk989898 opened this issue Apr 26, 2024 · 1 comment · May be fixed by #10990
Open

Enhance Kafka consumer performance #10989

wk989898 opened this issue Apr 26, 2024 · 1 comment · May be fixed by #10990
Assignees
@wk989898
Labels
area/ticdc Issues or PRs related to TiCDC. component/test Unit tests and integration tests component. kafka type/enhancement This is a enhancement PR

Comments

@wk989898
Copy link
Contributor

wk989898 commented Apr 26, 2024 •
edited

The following points need to be considered.

  • When the consumer crashes, some messages which haven't write to the downstream may be lost.
  • The code needs to be easy to maintain.
  • Reduce memory usage
@wk989898 wk989898 linked a pull request Apr 26, 2024 that will close this issue
Open
@3AceShowHand 3AceShowHand added component/test Unit tests and integration tests component. type/enhancement This is a enhancement PR area/ticdc Issues or PRs related to TiCDC. kafka labels Apr 26, 2024
@fubinzh
Copy link

fubinzh commented Apr 28, 2024

Can we addresss the issue with multiple consumer running together as well? @3AceShowHand @wk989898

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wk989898 wk989898

Labels
area/ticdc Issues or PRs related to TiCDC. component/test Unit tests and integration tests component. kafka type/enhancement This is a enhancement PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cmd (ticdc): a better Kafka consumer wk989898/tiflow
3 participants
@3AceShowHand @fubinzh @wk989898