-
-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Clarification NoOffsetReset #692
Comments
Sorry, ConsumeResetOffset is a really confusion option that I poorly named when tagging 1.0 and now the library is stuck with it. NoResetOffset is meant to be a modifier after OffsetOutOfRange is encountered. It doesn't affect how you start consuming. Looking at the code, the requested timestamp is initialized to MinInt64, but with the way Kafka works, that's functionally equivalent to requesting with the special timestamp -2 ("give me the start offset"). If the requested timestamp is not -1 nor -2, then Kafka returns the first offset after the requested timestamp. So, if the requested timestamp is min int, Kafka will return the first offset. I think you want a separate new behavior, NoInitialOffset (or some good name). I'll see if this is easy and if so, squeeze it in the next release. I don't think this would be good to bundle with NoResetOffset, because that's specifically around resetting in the face of a single specific error. |
Hello, thank you the quick response. I am not very familiar with the details of I realize my intent might not be perfectly clear. Essentially I am looking to set the equivalent of |
Had to put this on the backburner for a bit, but I see what you need -- I'm adding a new offset method |
I am having some trouble understanding exactly how NoResetOffset() works.
I am working only with group consumers. My use case is that I want to set the default behaviour to not touch any offset if there is are no offset, and ideally error on startup to let the caller know they have to make a choice what to do.
Reading the docs I think
NoResetOffset()
gives a little conflicting information:franz-go/pkg/kgo/consumer.go
Line 69 in 351e7fa
So if I pass
kgo.ConsumeResetOffset(kgo.NoResetOffset())
and the consumerGroup does not exist from earlier, will it refuse to start or start conusming from beginning, ie. same asNewOffset().AtStart()
? In the second case, how can I stop that from happening and ideally signal the caller that they need to actively set a ResetOffset policy?The text was updated successfully, but these errors were encountered: