Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

acme-http01-edit-in-place is ignored when edit ingress resource - has to be re-added #6065

Open
papanito opened this issue May 12, 2023 · 14 comments · May be fixed by #6567
Open

acme-http01-edit-in-place is ignored when edit ingress resource - has to be re-added #6065

papanito opened this issue May 12, 2023 · 14 comments · May be fixed by #6567
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@papanito
Copy link

papanito commented May 12, 2023 •
edited

Describe the bug:

I face some the following issue on aks: Waiting for HTTP-01 challenge propagation: failed to perform self check GET request.

Based on the recommendation, I add the annotation Waiting for HTTP-01 challenge propagation: failed to perform self check GET request

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    acme.cert-manager.io/http01-edit-in-place: 'true'
    cert-manager.io/cluster-issuer: myissuer
    cert-manager.io/common-name: nginx-demo.intra
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/force-ssl-redirect: 'false'
    nginx.ingress.kubernetes.io/rewrite-target: /
    service.beta.kubernetes.io/azure-load-balancer-internal: 'true'
    service.beta.kubernetes.io/azure-load-balancer-internal-subnet: snet-ingress
  name: nginx-demo-ingress
  namespace: nginx-demo
spec:
  rules:
    - host: nginx-demo.intra
      http:
        paths:
          - backend:
              service:
                name: nginx-demo-service
                port:
                  number: 80
            path: /
            pathType: Prefix
  tls:
    - hosts:
        - nginx-demo.intra
      secretName: nginx-demo-tls

However, despite the annotation the additional ingress is still created.

Expected behaviour:

My ingress is edited in place rather an additional one is created

Steps to reproduce the bug:

Anything else we need to know?:

Environment details::

  • Kubernetes version: 1.23.8
  • Cloud-provider/provisioner: AKS
  • cert-manager version: 1.11.0
  • Install method: e.g. helm/static manifests

/kind bug
@jetstack-bot jetstack-bot added the kind/bug Categorizes issue or PR as related to a bug. label May 12, 2023
@papanito
Copy link
Author

Any idea if this is a bug or I do something wrong?

@SergeyCP
Copy link

Same issue here in AWS EKS with k8s 1.24

@papanito
Copy link
Author

somebody looking at it?

@jpdasma
Copy link

jpdasma commented Jul 7, 2023

@papanito

Can you try to delete the ingress and recreate it again with the annotation in place? I tested this and this seems to happen if you updated the ingress resource with annotation instead of deleting it and then creating it again.

@papanito
Copy link
Author

papanito commented Jul 7, 2023

Yes that worked @jpdasma thanks a lot

@jpdasma
Copy link

jpdasma commented Jul 7, 2023

So yeah, I think what happens is that cert-manager doesn't detect any changes for existing Ingress. So if we have an existing Ingress, and then added the annotation acme.cert-manager.io/http01-edit-in-place: 'true' without deleting the existing Ingress, cert-manager will still create a new temporary Ingress.

@papanito
Copy link
Author

papanito commented Jul 7, 2023

You saved my day, I never though in the first place to just re-create the resource. So still, I keep the issue open as I believe this changes should be detected...

@papanito papanito changed the title acme-http01-edit-in-place seems to be ignored acme-http01-edit-in-place is ignored when edit ingress resource - has to be re-added Jul 14, 2023
@artooro
Copy link

artooro commented Aug 2, 2023

I believe I'm dealing with this issue right now. Tried deleting the ingress and re-creating it, but cert-manager is still creating a cm-acme-http-solver despite the acme.cert-manager.io/http01-edit-in-place: "true" annotation being set.

Have any of you had to delete other resources besides the ingress such as the certificate itself, orders, challenges, etc?

@Teemu-A
Copy link

Teemu-A commented Aug 17, 2023

A spec.ingressClassName: nginx is needed along/aside the metadata.annotations."kubernetes.io/ingress.class": nginx.
Similar on GKE v1.25.10-gke.2700.

@tobilau
Copy link

tobilau commented Sep 5, 2023

I am facing the same issue, on-premise. k8s v1.24.8
The ingress was created a while ago, so previous certificate renewals worked fine. This time, the extra cm-acme-http-solver was created, but without any ingressClass in place. Thus, the renewal failed until I deleted and re-deployed the ingress.

@Teemu-A Do you have both annotations in place and does it fix the issue? I replaced all the kubernetes.io/ingress.class annotations with spec.IngressClassName due to the deprecation.

@Teemu-A
Copy link

Teemu-A commented Sep 5, 2023

The cm-acme gets stuck without spec.ingressClassName despite of annotations. (For us this happens on helm installed gitlab which has cert-manager in the chart/release. That's why I do not want to touch the chart templates.)
I added some automation to regularly find such (ingresses without ingressClassName and name containing cm-acme-http-solver) and patch the correct value in, and it works good enough.

kubectl patch "ing/${ing1}" --type=json \
    -p='[{"op": "add", "path": "/spec/ingressClassName", "value":"gitlab-nginx"}]'

@rickydjohn
Copy link

@papanito

Can you try to delete the ingress and recreate it again with the annotation in place? I tested this and this seems to happen if you updated the ingress resource with annotation instead of deleting it and then creating it again.

this fixed my issue. In fact, these annotations existed already

    acme.cert-manager.io/http01-edit-in-place: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
    nginx.ingress.kubernetes.io/rewrite-target: /

Deleting the ingress and recreating it as suggested here fixed the issue.

@mahouESPRIT
Copy link

Issuing certificate as Secret was previously issued by "Issuer.cert-manager.io/"

Any solution ?

anothertobi added a commit to anothertobi/cert-manager that referenced this issue Dec 21, 2023
@anothertobi
Update Certificates with Ingress annotations
8b20465 
This updates the annotations of a Certificate owned by an Ingress when
they are added or changed after the Certificate exists. Before they were
only updated when an unrelated change, like a changed label value,
triggered an update.

Also removing the call to `setIssuerSpecificConfig()` as this is already
done before the `existingCrt` check.

Fixes cert-manager#6065

Signed-off-by: Tobi Nehrlich <tobi.nehrlich@amazee.io>
@anothertobi anothertobi linked a pull request Dec 21, 2023 that will close this issue
Open
anothertobi added a commit to anothertobi/cert-manager that referenced this issue Dec 21, 2023
@anothertobi
Update Certificates with Ingress annotations
4d7672c 
This updates the annotations of a Certificate owned by an Ingress when
they are added or changed after the Certificate exists. Before they
could only be "added" by deleting the Certificate itself.

Also removing the call to `setIssuerSpecificConfig()` as this is already
done before the `existingCrt` check.

Fixes cert-manager#6065

Signed-off-by: Tobi Nehrlich <tobi.nehrlich@amazee.io>
@vignan-devops
Copy link

Added the below annotations:
acme.cert-manager.io/http01-edit-in-place: 'true'
cert-manager.io/cluster-issuer:

spec:
ingressClassName: traefik

Still running into the same error

aks version : 1.27.7
cert-manager: 1.13.1
ingress controller : traefik

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update Certificates with Ingress annotations anothertobi/cert-manager
10 participants
@artooro @tobilau @rickydjohn @papanito @Teemu-A @jpdasma @jetstack-bot @SergeyCP @mahouESPRIT @vignan-devops