feat: raise alert on long running queries per user instead of single pid #29
Conversation
This commit modifies the existing alert on long running queries to be raised at the user level instead of the finer granularity of the pid level. The reason is that if a query holds a lock which blocks several queries, we will raise an alert for each query on hold, therefore spamming our alert channels and creating noise. Reducing the granularity to the user will improve readability and also understand quicker which apps and services are impacted.
|
#sre-storage |
|
@vmercierfr this would be a breaking change since I modified the alert name. How would you like to handle the release of such change? I would create a new release, but we should adhere to semantic versioning standards I guess |
| summary: "Long running query on {{ $labels.datname }} of {{ $labels.target }}" | ||
| description: "{{ $labels.usename }} is running a long query on {{ $labels.datname }} of {{ $labels.target }} with pid {{ $labels.pid }}" | ||
| summary: "Long running queries on {{ $labels.datname }} of {{ $labels.target }} initiated by {{ $labels.usename }}" | ||
| description: "{{ $labels.usename }} is running long queries on {{ $labels.datname }} of {{ $labels.target }}" |
There was a problem hiding this comment.
Wouldn't it be useful to keep the pid in the description as well?
This way you'd have everything you need right away to run a pg_cancel_backend or pg_terminate_backend.
WDYT?
There was a problem hiding this comment.
In this setup there are potentially several pids, to be honest I'm unsure if I can access them, and how I could properly display it. But yes, if possible it'd be nice to have them all in the description I guess!
LGTM, we would release a v0.2.0-beta.1 to be consistent
Meanwhile, I documented how to release versions: #30 |
dcupif
deleted the
group-alerts-for-long-running-queries-to-avoid-alert-spam
branch
This commit modifies the existing alert on long running queries to be raised at the user level instead of the finer granularity of the pid level.
The reason is that if a query holds a lock which blocks several queries, we will raise an alert for each query on hold, therefore spamming our alert channels and creating noise.
Reducing the granularity to the user will improve readability and also understand quicker which apps and services are impacted.