Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create rule only for the api #95

Open
Meza1923 opened this issue Dec 28, 2022 · 4 comments
Open

create rule only for the api #95

Meza1923 opened this issue Dec 28, 2022 · 4 comments
Assignees
Labels

Comments

@Meza1923
Copy link

Hey

I just installed the chart on my ocp cluster, and I want only to monitor the crt of the api.
Could someone help me achieve my goal?
Until now I set prometeusrule.create = false in order to create my own prometeusrules.
and I created with this configuration:

- alert: certificate
   rules: 
   - alert: certificate-expired
      annotations:
        description: Certificate of "{{ $labels.subject_CN }}" is about to end in {{ $value }} days
                {{if $labels.secret_name }}in Kubernetes secret "{{ $labels.secret_namespace
                }}/{{ $labels.secret_name }}"{{else}}
        summary: Certificate is about to expire
      expr: ((x509_cert_not_after - time()) / 86400) < 30
      for: 1h
      labels:
          severity: critical
@npdgm npdgm self-assigned this Dec 28, 2022
@npdgm npdgm added the usage label Dec 28, 2022
@npdgm
Copy link
Member

npdgm commented Dec 28, 2022

Hi,

I understand what you have done and your goal using this custom PrometheusRule. However you didn't describe what problem you encounter. I guess the alert is not working while you have certificates currently under the expiration threshold.

Before working on that it is necessary to confirm the exporter is deployed in an appropriate way for your k8s distribution and monitoring stack. OpenShift sometimes require more care and understanding as the default configuration is more elaborate in some ways than other typical distributions.
I don't think any developer on this project is also an OCP user so we're unlikely to guess what the issue may be right away. So let's go through the typical troubleshooting steps.

Could you please clarify if it is confirmed the exporter is being scrapped, and certificate metrics have appeared in Prometheus?
Going to the Prometheus web UI will help:

  • In the Status / Targets page there should entries named x509, that have been populated by prometheus-operator thanks to the ServiceMonitor or any other discovery method you may have enabled instead in chart values. Are they showing up green and UP?
  • Sending an expression such as x509_cert_not_after in the Graph form should list at least a few certificates.

If you cannot find x509 at all in the Targets list, then the issue has to do with how prometheus-operator is configured in the Prometheus object.
If the target was there but showing as DOWN, communication may be blocked by a NetworkPolicy which would not suprise me on OCP.

Please provide more information based on these points and questions, and what you did check already.

@Meza1923
Copy link
Author

Sorry you are right, My problem is when I created this prometeusrules, it creates around 300 alerts, and all I want is just ine alert on the api.
I can show the mertics you have write.

Thank you very much about your response.

@npdgm
Copy link
Member

npdgm commented Dec 29, 2022

Ok I get it now, you are looking for a set of labels to apply as a filter on the x509_cert_not_after metric. For your alert expression to only target specific certificates.

What is this "api" you are mentioning? Is it the Kubernetes api-server for the cluster?
Usually you would not find control-plane certificates stored in Secrets. Did you also enable the hostpathExporters?

@Meza1923
Copy link
Author

Meza1923 commented Jan 5, 2023

So sorry about the delay, I was out of office.
Yes, I meant to the Kubernetes api-server, In OpenShift we assimilating the api certificate in a secret.
Do you know any option to filter the x509_cert_not_after metric, in order to create specific alert?

Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants