Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict group_id label cardinality #6537

Merged
merged 2 commits into from May 16, 2024
Merged

Restrict group_id label cardinality #6537

merged 2 commits into from May 16, 2024

Conversation

bduffany
Copy link
Member

@bduffany bduffany commented May 9, 2024

The "Action stage durations" chart cardinality is too high and makes it slow to load. A single query loads over 300K individual timeseries from the DB.

Restricting to just prober group IDs should fix this (we want to split out prober latency to keep the prober latency chart working). If we need to query by other group IDs, we can use Clickhouse instead.

Related issues: N/A

@bduffany bduffany changed the title Remove group_id label from action stage durations chart Remove group_id label from action stage durations metric May 9, 2024
@bduffany bduffany marked this pull request as draft May 9, 2024 18:44
@bduffany
Copy link
Member Author

bduffany commented May 9, 2024

This would break the prober latency graph, need to think of how to keep that working.

@bduffany bduffany force-pushed the rm-group-id branch 2 times, most recently from 3e75a10 to 99dafed Compare May 9, 2024 19:20
@bduffany bduffany changed the title Remove group_id label from action stage durations metric Restrict group_id label cardinality to groups we care about May 9, 2024
@bduffany bduffany marked this pull request as ready for review May 9, 2024 19:32
@bduffany bduffany changed the title Restrict group_id label cardinality to groups we care about Restrict group_id label cardinality May 9, 2024

// GroupIDLabel returns a value suitable for use with the GroupID label. It
// restricts the observed group IDs to the allowed values configured by flag.
func GroupIDLabel(groupID string) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we call this FilteredGroupIDLabel? There will still be cases where we will want unfiltered group IDs on metrics.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@bduffany bduffany merged commit 7aea83d into master May 16, 2024
18 checks passed
@bduffany bduffany deleted the rm-group-id branch May 16, 2024 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants