-
-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Produce batch time metrics #704
Comments
What span of time are you attempting to capture, and with what information? Do you want to capture from the moment a record enters Produce through ... when? If you want to capture failures, do you mean for the batch duration to be called on every failure&retry, or only the final failure at which point the promise is called? |
Yeah I had a look at produce batch metrics and thought it was the wrong place. More interested on the Kafka request time then when a message gets put on the produce queue as this is when the I am not interested in capturing failures at the moment, but maybe its worth tracking these somehow generally, as I like the hook system as its useful to switch in and out different tools but maybe making harder to add error case handling. |
So, do you want essentially the time that a record was in the client? If so, couldn't you set r.Context before producing with a key indicating "producing now", and then check time.Since that key once the promise is called? |
To be able to see how long it took to to produce a batch to pass to monitoring systems to work out if producing timeout is to low or Kafka is being overloaded.
Expired issues where batch's get stuck in a retry loop because the batch is timing out and no way to diagnose this.
The text was updated successfully, but these errors were encountered: