Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Move offset validation logic to consumer classes" #13015

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

KKcorps
Copy link
Contributor

@KKcorps KKcorps commented Apr 26, 2024

Depends on - #12806

Copy link
Contributor

@Jackie-Jiang Jackie-Jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no guarantee we can always detect gap with start offset and batch first offset, especially after we introduce the exclusive start offset concept.
Seems like we still want to report gap within the general handling flow. In that case it should be easier to add an API hasGap() into MessageBatch, which can be filled by the stream consumer

@KKcorps
Copy link
Contributor Author

KKcorps commented Apr 29, 2024

There is no guarantee we can always detect gap with start offset and batch first offset, especially after we introduce the exclusive start offset concept. Seems like we still want to report gap within the general handling flow. In that case it should be easier to add an API hasGap() into MessageBatch, which can be filled by the stream consumer

That makes sense since we already have the information in the consumer from which offset did we actually wanted to consume the data vs from which offset we actually did OR any other criteria which consumer wants to implement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants