oqs-epiphany not working

[baentsch]

          Thanks, @jellejurre for the setup above. I'm absolutely baffled by what I see: The httpd server in your setup properly responds to queries fielded by an equivalent `oqsprovider` build/client, e.g.,

docker run --network epiphany_default -it openquantumsafe/curl sh -c "openssl s_client --connect epiphany-server-1:4433 -groups kyber768"

Also, the epiphany (browser) image in your composition properly operates openssl and oqsprovider when logging in to it via docker exec, e.g.

openssl list -providers -verbose
Providers:
  default
    name: OpenSSL Default Provider
    version: 3.3.0
    status: active
    build info: 3.3.0-dev
    gettable provider parameters:
      name: pointer to a UTF8 encoded string (arbitrary size)
      version: pointer to a UTF8 encoded string (arbitrary size)
      buildinfo: pointer to a UTF8 encoded string (arbitrary size)
      status: integer (arbitrary size)
  oqsprovider
    name: OpenSSL OQS Provider
    version: 0.5.4-dev
    status: active
    build info: OQS Provider v.0.5.4-dev (510fea6) based on liboqs v.0.10.0-dev
    gettable provider parameters:
      name: pointer to a UTF8 encoded string (arbitrary size)
      version: pointer to a UTF8 encoded string (arbitrary size)
      buildinfo: pointer to a UTF8 encoded string (arbitrary size)
      status: integer (arbitrary size)
openssl s_client -connect test.openquantumsafe.org:6036 -groups kyber768

"Just" when putting the two things together, i.e., using the known-good local openssl code against the known-good oqs-httpd, things fail:

openssl s_client -connect epiphany-server-1:4433 -groups kyber768
Connecting to 172.18.0.3
CONNECTED(00000003)
40F761B92B7F0000:error:0A000410:SSL routines:ssl3_read_bytes:ssl/tls alert handshake failure:ssl/record/rec_layer_s3.c:907:SSL alert number 40
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 1425 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)

oqs-epiphany then not working is a logical consequence. But the issue is somewhere else. Will need to debug into this...

Originally posted by @baentsch in #265 (comment)

[baentsch]

Local build of oqs-epiphany is not correctly working. Neither is the latest (0.9.2) image at docker hub. This requires a serious review of the changes in the underlying code base, most likely glib-networking.

[isaleh-sb]

Came here to make this ticket, but see that it's already reported 👍 ...

Just as an extra data point, the only openquantumsafe/epiphany image that I'm able to get any successful handling of PQC certificates with is 0.7.2 (this is hitting sites with Dilithium & Falcon certs hosted by the latest openquantumsafe/nginx image). 0.8.0 and onwards all fail with any PQC certs for every algorithm:

SSL_do_handshake() failed (SSL: error:0A000076:SSL routines::no suitable signature algorithm) while SSL handshaking

[baentsch]

Yup - I'm painfully aware of this. When looking into this, I got stymied by a crash of the docker image on my machine and have not found the time to look into this more deeply. If you'd be able to do some "digging", I'd be really grateful, @isaleh-sb . In general, running the command openssl s_client .... successfully is the prerequisite to epiphany working OK: Can you confirm (or the opposite :) of this being the case in your setup?

[jellejurre]

I don't know much about build systems but I did figure out the following:

RUN cd /opt/liboqs && mkdir build && cd build && cmake -GNinja -DOQS_DIST_BUILD=ON -DCMAKE_INSTALL_PREFIX=/opt/liboqs .. && ninja && ninja install
RUN cd /opt/oqs-provider && cmake -GNinja -DOQS_DIST_BUILD=ON -Dliboqs_DIR=/opt/liboqs -S . -B _build && cd _build && ninja && ninja install
RUN cd /opt/openssl && ./config no-shared --prefix=/opt/oqssa -lm && make -j 2  && make install_sw install_ssldirs && sed -i "s/default = default_sect/default = default_sect\noqsprovider = oqsprovider_sect/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/\[default_sect\]/\[default_sect\]\nactivate = 1\n\[oqsprovider_sect\]\nactivate = 1\n/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/providers = provider_sect/providers = provider_sect\nssl_conf = ssl_sect\n\n\[ssl_sect\]\nsystem_default = system_default_sect\n\n\[system_default_sect\]\nGroups = \$ENV\:\:DEFAULT_GROUPS\n/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/\# Use this in order to automatically load providers/\# Set default KEM groups if not set via environment variable\nKDEFAULT_GROUPS = $DEFAULT_GROUPS\n\n# Use this in order to automatically load providers/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/HOME\t\t\t= ./HOME\t\t= .\nDEFAULT_GROUPS\t= ${DEFAULT_GROUPS}/g" /opt/oqssa/ssl/openssl.cnf

doesn't work, yet

RUN cd /opt/liboqs && mkdir build && cd build && cmake -G"Ninja" .. -DOQS_DIST_BUILD=ON -DCMAKE_INSTALL_PREFIX=/opt/oqssa && ninja install
RUN cd /opt/openssl &&LDFLAGS="-Wl,-rpath -Wl,/opt/oqssa/lib64" ./config shared --prefix=/opt/oqssa && make -j 4 && make install_sw install_ssldirs;
RUN cd /opt/oqs-provider && ln -s ../openssl . && cmake -DOPENSSL_ROOT_DIR=/opt/oqssa -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=/opt/oqssa -S . -B _build && cmake --build _build  && cp _build/lib/oqsprovider.so /opt/oqssa/lib64/ossl-modules && sed -i "s/default = default_sect/default = default_sect\noqsprovider = oqsprovider_sect/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/\[default_sect\]/\[default_sect\]\nactivate = 1\n\[oqsprovider_sect\]\nactivate = 1\n/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/providers = provider_sect/providers = provider_sect\nssl_conf = ssl_sect\n\n\[ssl_sect\]\nsystem_default = system_default_sect\n\n\[system_default_sect\]\nGroups = \$ENV\:\:DEFAULT_GROUPS\n/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/\# Use this in order to automatically load providers/\# Set default KEM groups if not set via environment variable\nKDEFAULT_GROUPS = $DEFAULT_GROUPS\n\n# Use this in order to automatically load providers/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/HOME\t\t\t= ./HOME\t\t= .\nDEFAULT_GROUPS\t= ${DEFAULT_GROUPS}/g" /opt/oqssa/ssl/openssl.cnf

does.

I don't know enough about build configurations to know the difference, but maybe this is useful to you.

[jellejurre]

I've got it!

The openssl version that is installed before liboqs is installed needs to be 3.2 or higher. 3.1 or lower (the default ubuntu libssl-dev is 3.0.3 I believe) doesn't work.

To illustrate, this doesn't work:

FROM ubuntu as intermediate
ARG LIBOQS_TAG=main
ARG OQSPROVIDER_TAG=main
ENV DEBIAN_FRONTEND noninteractive

RUN apt update && apt upgrade -y && apt install -y build-essential clang meson gnome-pkg-tools libglib2.0-dev libproxy-dev ca-certificates libtool make gcc ninja-build cmake libtool wget libssl-dev
WORKDIR /opt
RUN git clone --branch ${LIBOQS_TAG} https://github.com/open-quantum-safe/liboqs.git && git clone --depth 1 --branch openssl-3.1 https://github.com/openssl/openssl && git clone --depth 1 --branch ${OQSPROVIDER_TAG} https://github.com/open-quantum-safe/oqs-provider.git

RUN cd /opt/openssl &&LDFLAGS="-Wl,-rpath -Wl,/opt/oqssa/lib64" ./config shared --prefix=/opt/oqssa && make -j 32 && make install_sw install_ssldirs;
RUN cd /opt/liboqs && mkdir build && cd build && cmake -GNinja -DOQS_DIST_BUILD=ON -DCMAKE_INSTALL_PREFIX=/opt/oqssa .. && ninja install
RUN cd /opt/oqs-provider && cmake -GNinja -DOQS_DIST_BUILD=ON -DCMAKE_PREFIX_PATH=/opt/oqssa -S . -B _build && cd _build && ninja && ninja install && cp lib/oqsprovider.so /opt/oqssa/oqsprovider.so
# RUN cp /opt/openssl/openssl.cnf /opt/oqssa/ssl/openssl.cnf
RUN sed -i "s/default = default_sect/default = default_sect\noqsprovider = oqsprovider_sect/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/\[default_sect\]/\[default_sect\]\nactivate = 1\n\[oqsprovider_sect\]\nactivate = 1\n/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/providers = provider_sect/providers = provider_sect\nssl_conf = ssl_sect\n\n\[ssl_sect\]\nsystem_default = system_default_sect\n\n\[system_default_sect\]\nGroups = \$ENV\:\:DEFAULT_GROUPS\n/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/\# Use this in order to automatically load providers/\# Set default KEM groups if not set via environment variable\nKDEFAULT_GROUPS = $DEFAULT_GROUPS\n\n# Use this in order to automatically load providers/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/HOME\t\t\t= ./HOME\t\t= .\nDEFAULT_GROUPS\t= ${DEFAULT_GROUPS}/g" /opt/oqssa/ssl/openssl.cnf

ENV PATH="/opt/oqssa/bin:${PATH}"
ENV OPENSSL_CONF=/opt/oqssa/ssl/openssl.cnf
ENV OPENSSL_MODULES=/opt/oqssa 
CMD openssl s_client -connect test.openquantumsafe.org:6104

yet the same script with branch openssl-3.2 does. In a similar vein, moving the openssl install after the oqs provider install breaks it as well, so this seems to imply it's the version of openssl that is used to install the oqs-provider that matters.

So by swapping my oqs install commands to the ones shown above (+ some other config), I managed to get my dockerfile to handle signatures correctly.

Maybe it would be smart to use the fullbuild script in the oqs-epiphany demo to circumvent this

[baentsch]

I've got it!

Thanks for letting us know!

Maybe it would be smart to use the fullbuild script in the oqs-epiphany demo to circumvent this

Sounds like a good suggestion. Would you feel like contributing a PR to facilitate that?

[jellejurre]

I'd love to, but I can't actually run the default image so I can't test it ^^

The only linux machines I have are on VMs and those dont like the display streaming through docker that this uses

[jellejurre]

Oh and this change did manage to make openssl work on my branch, but epiphany still didnt do the signatures properly for some reason

[baentsch]

Oh and this change did manage to make openssl work on my branch, but epiphany still didnt do the signatures properly for some reason

Now I'm confused: Didn't you say before signatures work OK with 3.2:

yet the same script with branch openssl-3.2 does. In a similar vein, moving the openssl install after the oqs provider install breaks it as well, so this seems to imply it's the version of openssl that is used to install the oqs-provider that matters.

So by swapping my oqs install commands to the ones shown above (+ some other config), I managed to get my dockerfile to handle signatures correctly.

The latter is (for openssl) entirely expected: See https://github.com/open-quantum-safe/oqs-provider?tab=readme-ov-file#note-on-openssl-versions (some provider-sig-enabling PRs didn't get backported to older OpenSSL versions).

[jellejurre]

yeah, so openssl did the signatures correctly, but epiphany did not.

This could be because my fix was bad, as I am not an expert on the networking epiphany uses.

I'll see if I can get a dockerfile that shows this.

[jellejurre]

yeah I think I might be putting the provider in the wrong place or something. This is my code where openssl works fine but the epiphany doesn't.

# define the liboqs tag to be used
ARG LIBOQS_TAG=main

# define the oqsprovider tag to be used
ARG OQSPROVIDER_TAG=main

# Default location where all binaries wind up:
ARG OSSLGIODIR=/opt/ossl-gio
ARG INSTALLDIR_OPENSSL=/opt/ossl-gio/openssl

# 2-step build: First create binaries
FROM ubuntu as intermediate
ARG LIBOQS_TAG
ARG OQSPROVIDER_TAG
ARG OSSLGIODIR
ARG INSTALLDIR_OPENSSL
ENV DEBIAN_FRONTEND noninteractive

# everything to build liboqs, oqs-openssl, glib-networking:

RUN apt update && apt upgrade -y && apt install -y build-essential clang meson gnome-pkg-tools libglib2.0-dev libproxy-dev gsettings-desktop-schemas-dev ca-certificates epiphany-browser libtool make gcc ninja-build libssl-dev cmake libtool wget

WORKDIR /opt

RUN git clone --depth 1 --branch openssl-3.2 https://github.com/openssl/openssl && git clone --depth 1 --branch ${OQSPROVIDER_TAG} https://github.com/open-quantum-safe/oqs-provider.git && git clone https://gitlab.gnome.org/GNOME/glib-networking.git
WORKDIR /opt/openssl
RUN LDFLAGS="-Wl,-rpath -Wl,${INSTALLDIR_OPENSSL}/lib64" ./config shared --prefix=${INSTALLDIR_OPENSSL} && \
    make ${MAKE_DEFINES} && make install && if [ -d ${INSTALLDIR_OPENSSL}/lib64 ]; then ln -s ${INSTALLDIR_OPENSSL}/lib64 ${INSTALLDIR_OPENSSL}/lib; fi && if [ -d ${INSTALLDIR_OPENSSL}/lib ]; then ln -s ${INSTALLDIR_OPENSSL}/lib ${INSTALLDIR_OPENSSL}/lib64; fi 

ENV OPENSSL_ROOT_DIR="${INSTALLDIR_OPENSSL}"
RUN cd /opt/oqs-provider && ./scripts/fullbuild.sh && cp _build/lib/oqsprovider.so ${OSSLGIODIR}/oqsprovider.so  && cp _build/lib/oqsprovider.so ${INSTALLDIR_OPENSSL}/lib64/ossl-modules
RUN cp /opt/openssl/apps/openssl.cnf ${OSSLGIODIR}/openssl.cnf
RUN sed -i "s/default = default_sect/default = default_sect\noqsprovider = oqsprovider_sect/g" ${OSSLGIODIR}/openssl.cnf && sed -i "s/\[default_sect\]/\[default_sect\]\nactivate = 1\n\[oqsprovider_sect\]\nactivate = 1\n/g" ${OSSLGIODIR}/openssl.cnf && sed -i "s/providers = provider_sect/providers = provider_sect\nssl_conf = ssl_sect\n\n\[ssl_sect\]\nsystem_default = system_default_sect\n\n\[system_default_sect\]\nGroups = \$ENV\:\:DEFAULT_GROUPS\n/g" ${OSSLGIODIR}/openssl.cnf && sed -i "s/\# Use this in order to automatically load providers/\# Set default KEM groups if not set via environment variable\nKDEFAULT_GROUPS = $DEFAULT_GROUPS\n\n# Use this in order to automatically load providers/g" ${OSSLGIODIR}/openssl.cnf && sed -i "s/HOME\t\t\t= ./HOME\t\t= .\nDEFAULT_GROUPS\t= ${DEFAULT_GROUPS}/g" ${OSSLGIODIR}/openssl.cnf

# build glib-networking such as to use (OQS-)OpenSSL and not GnuTLS:
RUN cd /opt/glib-networking && git checkout 2.72.2 && mkdir build && cd build && PKG_CONFIG_PATH=${OSSLGIODIR}/lib/pkgconfig CPATH=${OSSLGIODIR}/include LIBRARY_PATH=${OSSLGIODIR}/lib  meson --prefix=${OSSLGIODIR} -Dopenssl=enabled -Dgnutls=disabled .. && CPATH=${OSSLGIODIR}/include ninja && ninja install

COPY certlink.sh /usr/local/ssl/certs/certlink.sh
RUN cd /usr/local/ssl/certs && wget https://letsencrypt.org/certs/isrgrootx1.pem && wget https://test.openquantumsafe.org/CA.crt -O oqsrootca.pem && ./certlink.sh isrgrootx1.pem && ./certlink.sh oqsrootca.pem

# 2nd build step: Only retain what's necessary:
FROM ubuntu
ARG OSSLGIODIR
ARG INSTALLDIR_OPENSSL
ENV DEBIAN_FRONTEND noninteractive

RUN apt update && apt upgrade -y && apt install -y epiphany-browser 

COPY --from=intermediate ${OSSLGIODIR} ${OSSLGIODIR}
COPY --from=intermediate /usr/local /usr/local
COPY --from=intermediate ${OSSLGIODIR}/oqsprovider.so /usr/local/lib64/ossl-modules/oqsprovider.so
RUN ln -s /usr/lib/x86_64-linux-gnu/ossl-modules /usr/local/lib64/ossl-modules

# Run everything under a limited user account:
RUN groupadd -g 1000 oqs && useradd -u 1000 -d /home/oqs -g oqs oqs 
COPY openssl-client.cnf /home/oqs/openssl-client.cnf
RUN chown -R oqs.oqs /home/oqs

RUN apt-get update && \
    apt-get install -y wget gnupg2 apt-transport-https && \
    wget -O - https://xpra.org/gpg.asc | apt-key add - && \
    echo "deb https://xpra.org/ jammy  main" > /etc/apt/sources.list.d/xpra.list

RUN apt-get update && \
    apt-get install -y xpra

USER oqs

ENV PATH="${INSTALLDIR_OPENSSL}/bin:${PATH}"
ENV OPENSSL_CONF=${OSSLGIODIR}/openssl.cnf

ENV LD_LIBRARY_PATH=${OSSLGIODIR}/lib
ENV GIO_MODULE_DIR=${OSSLGIODIR}/lib/x86_64-linux-gnu/gio/modules

CMD WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS=1 /usr/bin/xpra start --start-child=epiphany --exit-with-children --bind-tcp=0.0.0.0:9876 --bell=no --mdns=no --notifications=no --resize-display=true --daemon=no

STOPSIGNAL SIGTERM

with this, docker run -it browsertest openssl s_client -connect test.openquantumsafe.org:6109 --groups kyber768 works but docker run -p 9876:9876 browsertest and going to test.openquantumsafe.org:6109 or test.openquantumsafe.org:6057 doesn't. Neither for kem not for signature, so if I had to guess I'm putting the oqs provider in the wrong places.