Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improvement] Suggestion for Handling Stale Metrics in Gravitino #3152

Open
TEOTEO520 opened this issue Apr 23, 2024 · 2 comments · May be fixed by #3341
Open

[Improvement] Suggestion for Handling Stale Metrics in Gravitino #3152

TEOTEO520 opened this issue Apr 23, 2024 · 2 comments · May be fixed by #3341
Assignees
@FANNG1
Labels
improvement Improvements on everything

Comments

@TEOTEO520
Copy link
Contributor

What would you like to be improved?

Issue Description
There is a current limitation in the gravitino_server_http_request_duration_seconds{operation="xxx", quantile="0.xx"} metrics within Gravitino. If an operation is not called for a prolonged period after its last execution, the metric continues to display the duration of the last call. This behavior can lead to misinterpretations of the system's performance, as it does not accurately reflect the inactivity.

Suggested Improvement
It would be beneficial to modify the metric behavior to either reset to zero or drop the metric data when no calls are made to an operation within a predefined timeout period. This change would provide a more accurate representation of the system's current state and activity levels.

image

@FANNG1 +cc

How should we improve?

No response
@TEOTEO520 TEOTEO520 added the improvement Improvements on everything label Apr 23, 2024
@FANNG1
Copy link
Contributor

FANNG1 commented Apr 24, 2024

thanks for reporting this issue.

@FANNG1 FANNG1 self-assigned this May 10, 2024
@FANNG1 FANNG1 linked a pull request May 11, 2024 that will close this issue
Open
@FANNG1
Copy link
Contributor

FANNG1 commented May 11, 2024

@TEOTEO520 , #3341 propose a way to compuate Pxx according to the time sliding window, you could check it when you are free. I'm not sure whether to make it configurable for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FANNG1 FANNG1

Labels
improvement Improvements on everything
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[#3152] feat(core): using time sliding window to record timer metrics FANNG1/gravitino
2 participants
@FANNG1 @TEOTEO520