Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SLOs in Prometheus #487

Open
d2k-klin opened this issue Mar 11, 2023 · 3 comments
Open

SLOs in Prometheus #487

d2k-klin opened this issue Mar 11, 2023 · 3 comments

Comments

@d2k-klin
Copy link

d2k-klin commented Mar 11, 2023

I have deployed latest slot version(v0.11.0) with Helm.

i have defined a PrometheusServiceLevel :

kind: PrometheusServiceLevel
metadata:
  name: sloth-example-profile
  namespace: monitoring
  labels:
    prometheus: prometheus
    role: alert-rules
    app: sloth  
spec:
  service: example-profile
  slos:
    - alerting:
        name: example_profile_availability_alert
        pageAlert:
          labels:
            severity: warning
        ticketAlert:
          labels:
            severity: info
      name: example_profile_availability
      objective: 99
      sli:
        events:
          errorQuery: count_over_time((sum(up{namespace="exampleplatform",pod =~ "example-profile.*"})<1)[{{.window}}:])
          totalQuery: count_over_time(sum(up{namespace="exampleplatform",pod =~ "example-profile.*"})[{{.window}}:])

the PrometheusRule is successfully generated:

NAME                 SERVICE        DESIRED SLOS   READY SLOS   GEN OK   GEN AGE   AGE
sloth-example-profile   example-profile   1              1            true     3m19s     99m

but the Prometheus is not finding metrics for 'slo':
image
And without this the Grafana dashboards are just empty.

Am i missing something? Is there any additional step for this?

@klubi
Copy link

klubi commented Mar 13, 2023

Maybe prometheus does not ingest PrometheusRules because either label or namespace selectors for rules are out of scope.

@alexanderjardim
Copy link

I am having the same problem with 0.11. @d2k-klin did you manage to solve the issue?

@emdneto
Copy link

emdneto commented Jul 31, 2023

@alexanderjardim Probably you're missing the label as @klubi said.

Look in your prometheus config for ruleSelector to confirm the label you're using.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants