cannot load certificate, Expecting: TRUSTED CERTIFICATE

[djuarezg]

Cannot run HTTPS on the 1.3.1 image due to the following startup error:

nginx-proxy_1  | nginx.1     | 2023/08/18 15:59:31 [error] 46#46: *1 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0909006C:PEM routines:get_name:no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: <IP>, server: 0.0.0.0:8881

This is the Docker compose to reproduce it:

version: '2'

services:
  nginx-proxy:
    image: nginxproxy/nginx-proxy:1.3.1
    network_mode: "host"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - /tmp/nginx:/etc/nginx/conf.d
      - /etc/pki/tls/certs/:/etc/nginx/certs/:ro
    environment:
      - HTTP_PORT=8879
      - HTTPS_PORT=8881


  whoami:
    image: jwilder/whoami
    network_mode: "host"
    environment:
      - VIRTUAL_HOST=dtapi.domain.com
      - VIRTUAL_PORT=8000
      - VIRTUAL_PATH=/whoami

  # Sample hello world web service serving on port 8001
  hello:
    image: tutum/hello-world
    network_mode: "host"
    environment:
      - VIRTUAL_HOST=dtapi.domain.com
      - VIRTUAL_PORT=80
      - VIRTUAL_PATH=/hello

For reference these are the crt and key files (REDACTED):

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            <REDACTED>
        Signature Algorithm: sha512WithRSAEncryption
        Issuer: CN=<REDACTED>
        Validity
            Not Before: Sep 13 10:23:24 2022 GMT
            Not After : Aug 28 10:23:24 2025 GMT
        Subject: CN=vmk-dtapi-01
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:<REDACTED>
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
            X509v3 Subject Key Identifier:
                <REDACTED>
            X509v3 Authority Key Identifier:
                keyid: <REDACTED>
                DirName:/CN=<REDACTED>
                serial:<REDACTED>

            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Key Usage:
                Digital Signature, Key Encipherment
            X509v3 Subject Alternative Name:
                DNS:vmk-dtapi-01, DNS:vmk-dtapi-01.domain.com, IP Address:
    Signature Algorithm: sha512WithRSAEncryption
-----BEGIN CERTIFICATE-----
<REDACTED>
-----END CERTIFICATE-----

-----BEGIN PRIVATE KEY-----
REDACTED
-----END PRIVATE KEY-----

Tried using a crt file produced with openssl x509 -in dtapi.domain.com.crt --trustout -out dtapi.domain.com.crt without any change of output.

[lemzoo]

Hello, I encounter currently the same issue

2023/08/21 15:31:51 [error] 40#40: *22 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 10.170.78.215, server: 0.0.0.0:443

[lemzoo]

I tested some images and I found all the new images build after this 1.2-alpine didn't works.
And all the image with the oldest tag works as expected

[jordonedavidson]

Can confirm that release 1.3.1 has this behaviour, while release 1.3.0 does not.

-- Correction --
1.3.0 still throws the error periodically in the logs but does not seem to prevent the loading of the secured site.

[djuarezg]

Found a workaround, which does not correspond to what the README states. If you name the crt and key files as "default.crt" or "default.key" as per https://github.com/nginx-proxy/nginx-proxy/blob/main/nginx.tmpl#L14C77-L14C77, it seems to work fine with the latest version.

So, either fix the template or update the README.

[icsy7867]

I ran into this as well. Creating a default.crt and default.key fixed the issue.

[jmvallejo]

Would this solution still support multiple virtual hosts with ssl? What's the latest version that doesn't have the issue? I'm running into the same problem, thanks!

[osnard]

I have the same issue with the latest version. Unfortunately using the default.pem name approach didn't work for me.

[osnard]

HINT: Apparently I did something wrong in the first place. With

openssl req -x509 -newkey rsa:4096 -keyout <hostname>.key -out <hostname>.crt -sha256 -days 3650 -nodes -subj "/C=...ST=.../L=.../O=.../OU=.../CN=..."

it worked.

[buchdag]

I've been unable to reproduce this, I've never had the issue with Let's Encrypt or ZeroSSL certificates, and I'm not certain what the issue is here. This error (PEM_read_bio_X509_AUX() failed) is OpenSSL telling you it can't read what should be a valid certificate file (valid in the sense encoded as expected by OpenSSL). Having or not having a default certificate should have no effect on wether OpenSSL can or cannot read another certificate 🤔

@osnard do you remember what command you used in the first place to produce the non working cert ?

[ortiz-kuakside]

hi all , using version 1.3.0 worked

[cicnavi]

I've encountered the same error while trying to use regular expression in the host together with wildcard certificate, with example compose file being below.

So, if I tried to enter a non-regular expression host (for example latest-dap.localhost.markoivancic.from.hr), it worked. If I tried to enter regular expression host (for example abc-latest-dap.localhost.markoivancic.from.hr), ti would error out with "Expecting: TRUSTED CERTIFICATE".

The solution which worked for me was to explicitly set the shared cert with CERT_NAME environment variable.

services:
    nginx-proxy:
        restart: always
        image: nginxproxy/nginx-proxy:latest
        ports:
            - 80:80
            - 443:443
        environment:
            DEFAULT_HOST: localhost.markoivancic.from.hr
        volumes:
            - ./nginx-proxy/certs:/etc/nginx/certs
            - /var/run/docker.sock:/tmp/docker.sock:ro

    08.dap.test:
        image: cicnavi/dap:08
        container_name: 08.dap.test
        restart: unless-stopped
        expose:
            - 80
            - 443
        user: ${CONTAINER_USER_ID}
        environment:
            VIRTUAL_HOST: >-
                localhost.markoivancic.from.hr,
                08.dap.test,
                latest-dap.localhost.markoivancic.from.hr,
                ~^.+-latest-dap\.localhost\.markoivancic\.from\.hr$,
                08-dap.localhost.markoivancic.from.hr,
                ~^.+-08-dap\.localhost\.markoivancic\.from\.hr$
            CERT_NAME: localhost.markoivancic.from.hr
            HTTPS_METHOD: noredirect
        volumes:
            - "./dap/08/html:/var/www/html"

[pkimtani]

I generated my self-signed certificate using mkcert, and I was facing the same issue (running version 1.4-alpine). I created a copy of the same crt and key files and renamed them to default.crt and default.key and everything started to work.

[postelrich]

Was there a resolution to this? Seeing

nginx.1     | 2024/03/10 04:48:37 [error] 509#509: *101 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 192.168.86.170, server: 0.0.0.0:443

with nginx-proxy:1.4. The certs are generated with certbot on the host machine and volume mapped with container.

sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials ~/.cloudflare.ini -d example.com -d *.example.com --rsa-key-size 2048 --key-type rsa

[buchdag]

@postelrich unfortunately there wasn't really any reproducible bug identified yet, see #2287 (comment)

That probably won't change until someone provide a config and a non redacted, non production and disposable cert / key pair (or full command to generate one) that allow to reproduce the issue.

[postelrich]

@buchdag after looking further, my issue is that nginx-proxy is not picking up the ssl cert/key no matter what naming I try. If exec into the container, I can see the ssl configs are missing. If I manually add them in for the upstream server, it works.

[buchdag]

Could you:

• post your full nginx-proxy and proxied app configuration (compose file or equivalent)
• post the result of docker exec yourproxycontainer ls -lh /etc/nginx/certs
• check if you can cat those certificates from inside the container (I'd say yes if manually adding them to the nginx config worked)

?

My guess at this point would be that you are mounting a host symlink inside the container.

[postelrich]

@buchdag it was a symlink since I'm using certbot. I think in my various attempts to get this working, I forgot that when I hardcoded I had mounted the linked directory too. Copying the generated pem files instead of symlinking to example.com.crt and example.com.key worked as expected. Thanks for the reminder.

[buchdag]

Glad you found the issue 👍

If I had to use certbot with nginx-proxy I think I would use the post renewal hook to put the certificates where they should be (with the correct name) and trigger docker-gen and/or reload the nginx config.

The recommended way to get TLS certificate from an ACME CA with nginx-proxy is still to use nginx-proxy/acme-companion.

[postelrich]

@buchdag yea I tried acme-companion first but these sites are LAN only so couldn't do http challenge.

[jordonedavidson]

I'm wondering if the network mode is part of the problem. In the original issue the network mode is host, which I believe binds the container to the host machine's IP address. I'm having a similar issue when using the driver_opts key in my networks definition where I have this:

networks:
  app-tier:
    ipam:
      config:
        - subnet: 192.168.100.0/24
    driver_opts:
      # This binds this network's ports to the ip address defined by the BIND_IP
      # environment variable.
      com.docker.network.bridge.host_binding_ipv4: ${BIND_IP}

The error in the logs shows that the proxy server upstream cannot get the value of "data(): 0.0.0.0:443" or something similar. The cert is included in the container in the appropriate location. Maybe this is something to check out?

I have nearly identical setups in local and testing setups, with this binding being the main difference. Only this setup produces the errors. Note i'm currently using version 1.5.1