Alerting support for queries using template variables in Grafana will be available "when"?

[fchiorascu]

Hi,

Sorry to open this topic.
I’ve tried to implement the alert in Grafana Dashboard but not working when you have a Dashboard with templating.
I’ve noticed that are some opened topics on GitHub regarding this subject: #6230, #6557.

–=Solution=–
The solution will be to have a Dashboard per Server in order to have alerting working?
vs
Other solution will be to have Dashboard per Services in order to have the alerting working.

Kind Regards,

[fchiorascu]

Ex of templating:

I understood that is not possible to have Alert in Grafana per each server present in the same Dashboard.
No resolution on internet regarding this, the only thing remains to have per service per server a Dashboard in order to have Alert available.

Any idea who to ask about this topic?
Below something on Internet: #6230

[fchiorascu]

What are my expectations to work this Dashboard with templating and Alert in Grafana is to have the same alert per 1 single Dashboard that has 50 servers there. For the rest of the cases where the WARNING/ CRITICAL thresholds must be different from server to server I think i could create Dashboard per server in order to work the alert or per service?

[torkelo]

dupe of #6557

The solution will be to have a Dashboard per Server in order to have alerting working?
vs
Other solution will be to have Dashboard per Services in order to have the alerting working.

The current solution we recommend is to use wildcards in your query so that it returns many series. No ETA on when some limited form of template variable support will be implemented (or if ever)

[fchiorascu]

Thank you for your intervention.
I understood.

[martinler]

@torkelo Just "reality-check": with"wildcard" solution there is limitations, which you described in past:

yes, it will check all series. But currently it does not keep state per series. So if one server has high CPU the alert rule will trigger (With the info on which server) and send out a notification, but a minute later another server also has high cpu it will not trigger new notifications as the alert rule is still in firing state.

So if i need to fire separate alerts for each unit, in best case with different rules (who will be notified), Grafana is not right tool for my needs and quite possible never will be, am i right?

Any idea/suggestion on existing solution?

Thank you!

[torkelo]

Keeping state per series is something we really want to do (hopefully within months).

[cybfly]

hope it can be solved

[ipolishchuk]

Any update on this? With multiple servers the existing Grafana alerting capabilities look insufficient enough, so we still need to keep nagios running just for alerting purposes.

[amjad]

dupe of #6557

The solution will be to have a Dashboard per Server in order to have alerting working?
vs
Other solution will be to have Dashboard per Services in order to have the alerting working.

The current solution we recommend is to use wildcards in your query so that it returns many series. No ETA on when some limited form of template variable support will be implemented (or if ever)

Can you share an example of how this can be done? Thanks.

[podstava]

Current solution for me was removing hostname veriable from dashboard and grouping by hostname in query.

[ybello33]

Any Update on this open issue ?

[sbilello]

It would be a great feature to have to save developer time to build panels

[hluaces]

Apologies for asking about a closed issue, but I couldn't find information about this neither on the changelog nor on any open issue.

#10412 (comment):

Keeping state per series is something we really want to do (hopefully within months).

@torkelo , could you tell us if this is still on the roadmap? any ETA for it?

Thanks to everyone for your work.

[ToWi87]

if each time series has a separate label (which it usually has if you're using template variables), everyone here might want to look into this:
#7832 (comment)
resp. https://grafana.com/docs/grafana/latest/alerting/unified-alerting/alerting-rules/create-grafana-managed-rule/#single-and-multi-dimensional-rule

[torkelo]

Keeping state per series is something we really want to do (hopefully within months).

this is part of the new alerting system introduced in v8