Using Connaisseur with cosign and private Azure Container Registry

[marese98]

Did anyone get this working? I have configured this according to the documentation, but always get an UNAUTHORIZED error from ACR:

Error from server: error when creating "STDIN": admission webhook "connaisseur-svc.connaisseur.svc" denied the request: Unexpected Cosign exception for image "maresetest.azurecr.io/ifs/example-app:latest": Error: GET https://maresetest.azurecr.io/oauth2/token?scope=repository%3Aifs%2Fexample-app%3Apull&service=maresetest.azurecr.io: UNAUTHORIZED: authentication required, visit https://aka.ms/acr/authorization for more information.
main.go:69: error during command execution: GET https://maresetest.azurecr.io/oauth2/token?scope=repository%3Aifs%2Fexample-app%3Apull&service=maresetest.azurecr.io: UNAUTHORIZED: authentication required, visit https://aka.ms/acr/authorization for more information.

This is my values.yaml:

---

application:
validators:
# IFS validator
- name: ifs
type: cosign
trustRoots:
- name: ifs
key: |
-----BEGIN PUBLIC KEY-----
#####################
-----END PUBLIC KEY-----
auth.SecretName: my-secret
policy:
- pattern: "maresetest.azurecr.io/:"
validator: ifs
with:
trustRoot: ifs

I have created the necessary my-secret in connaisseur namespace as per documentation, but still no luck.

I noticed that when using notary you are supposed to add isAcr: true, which perhaps is an indication ACR is a bit 'special'?

[Starkteetje]

Hi @marese98

what version of Connaisseur are you using? Cosign seems to have had such an issue in the past, so in case you're using an outdated Connaisseur version, the bundled Cosign version might not contain the upstream fixes.

Otherwise: the ACR does indeed do things differently (e.g. not implementing all parts of the Notary server interface though claiming compatibility or using special auth token names), so the likelihood for there to be a problem (either on our side or upstream) is indeed not insignificant. If it's not due to an outdated version, I'll go digging a bit.

Kind regards
Teetje

[marese98]

I have tried two ways of installing, first by cloning https://github.com/sse-secure-systems/connaisseur.git and doing a local helm install.
I also tried the helm charts published here: https://sse-secure-systems.github.io/connaisseur/charts

Both give me the same result.

[phbelitz]

Hoy @marese98

To clarify the confusion with isAcr. That is only needed for validation using a notary server (meaning using Dokcer Content Trust). That isn't the case for you, so you shouldn't need to use that field. Also to clarify things here: the ACR's notary server sends back its authentication token in a different field then the default notary server (the one used by Docker themselves) would do. That's why this isAcr field exists.

Now to your problem. I managed to get things going using an ACR, so I'm gonna run you through my process, so you can replicate things or pinpoint issues:

1. Created a container registry in my azure instance
2. Did az login followed by az acr login --name <ACR-NAME> to have the credentials for pushing and signing
3. Pushed and signed a sample image into the ACR using cosign
4. I created a service pricipal for my registry, so I have a simple username/password combination to communicate with the ACR:

# Retrieve the ID of your registry
REGISTRY_ID=$(az acr show --name <ACR-NAME>  --query 'id' -otsv)

# Create a service principal with the Reader role on your registry
az ad sp create-for-rbac --name "<SERVICE-PRINCIPLE-NAME>" --role Reader --scopes ${REGISTRY_ID}

5. The resulting service pricipal has a applicationID and password, which I used for creating an authentication secret:

kubectl create secret docker-registry <YOUR_SECRET-NAME> --docker-server=<ACR-URL> --docker-username=<APPLICATION-ID> --docker-password=<PASSWORD>

6. Configured Connaisseur to use the secret for a cosign validator:

application:
  validators:
  - name: acr-test
    type: cosign
    trustRoots:
    - name: default
       key: |
         ...
    auth:
      secretName: <YOUR_SECRET-NAME>
  policy:
  - pattern: <ACR-URL>/*:*
    validator: acr-test
    with:
      verifyInTransparencyLog: false

7. Successfully validated the image inside my local minikube cluster

Hopefully this helps.

Cheers.

[marese98]

Thanks, that got me a bit further (I think I had created the secret in the wrong way).
Now, when attempting to deploy I instead get the following error:

Error from server: error when creating "STDIN": admission webhook "connaisseur-svc.connaisseur.svc" denied the request: Unexpected Cosign exception for image "maresetest.azurecr.io/ifs/example-app:latest": WARNING: Skipping tlog verification is an insecure practice that lacks of transparency and auditability verification for the signature.
Error: Get "https://maresetest.azurecr.io/v2/": dial tcp: lookup maresetest.azurecr.io on 10.43.0.10:53: server misbehaving
main.go:69: error during command execution: Get "https://maresetest.azurecr.io/v2/": dial tcp: lookup maresetest.azurecr.io on 10.43.0.10:53: server misbehaving

[phbelitz]

that's a DNS issue and should be unrelated to Connaisseur. are you by chance using minikube in combination with WSL ?

[marese98]

Almost. k3d and WSL.
But my cluster is able to pull the image from ACR so perhaps not a general DNS issue.

I'll test this using AKS instead.

[marese98]

OK, so this works fine outside of WSL (tested using AKS).
Thanks, for your help, @phbelitz!

[phbelitz]

Yea. Try to change your nameserver inside you WSL. Edit the /etc/resolv.conf and change the nameserver to e.g. 8.8.8.8. Then your local setup will probably work as well.

This error message seemed very familiar 😄