Failing to run fipsinstall on HP-UX

[sanchit-rubrik]

Summary

We are building OpenSSL 3.0.7 on an HP-UX ia64 11.31 machine (Build Machine), and using the generated fips.so file to generate a FIPS configuration file on a different HP-UX machine (User Machine; also ia64 11.31). When we try to run openssl fipsinstall on the User Machine, it fails with Unable to find library 'libgcc_s.so.0'.

Does the fips.so file have a dependancy on gcc? Note that we had built OpenSSL using gcc, but the User Machine (the one on which we are running openssl fipsinstall) doesn't have gcc. It only has the bundled hpux cc compiler.

Details on how we built OpenSSL 3.0.7

Details on the Build Machine

$ uname -a
HP-UX s-1537-v B.11.31 U ia64 HP-UX

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
Target: ia64-hp-hpux11.31
Configured with: ../gcc-4.8.5/configure --prefix=/opt/gcc-4.8.5 --enable-checking=release --with-gnu-as --with-as=/usr/local/bin/as --without-gnu-ld --with-ld=/usr/ccs/bin/ld --enable-languages=c,c++ --enable-shared --disable-nls --disable-libgcj --disable-libgomp --with-dwarf2 --enable-threads=posix --enable-targets=all CC='gcc -mlp64 -D_HPUX_SOURCE -D_INCLUDE_HPUX_SOURCE' CXX='g++ -mlp64 -D_HPUX_SOURCE -D_INCLUDE_HPUX_SOURCE' LDFLAGS=-mlp64 CFLAGS='-O2 -g -mlp64' CXXFLAGS='-O2 -g -mlp64' --disable-bootstrap
Thread model: posix
gcc version 4.8.5 (GCC)

Commands used to build OpenSSL

$ cd openssl-3.0.7
$ export CC=gcc; export CXX=g++; export CFLAGS="-pthread -O2 -g -mlp64 -D_GLIBCXX_USE_C99"; export CXXFLAGS="-pthread -O2 -g -mlp64 -D_GLIBCXX_USE_C99 -std=c++11"
$ ./Configure --prefix=/openssl_build_dir --openssldir=/openssl_build_dir hpux64-ia64-gcc no-shared enable-fips
$ gmake -j4
$ gmake install -j4

Details on running fipsinstall

Then we copied the generated openssl_build_dir/bin/openssl binary and openssl_build_dir/lib/hpux64/ossl-modules/fips.so file to the User Machine, and ran this command:

$ /path/to/openssl fipsinstall -module /path/to/fips.so -out openssl-fips.cnf -provider_name fips
Failed to load FIPS module
INSTALL FAILED
00000001:error:12800067:DSO support routines:(unknown function):could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/ossl-modules/fips.so): Unable to find library 'libgcc_s.so.0'.
00000001:error:12800067:DSO support routines:(unknown function):could not load the shared library:crypto/dso/dso_lib.c:152:
00000001:error:07880025:common libcrypto routines:(unknown function):reason(524325):crypto/provider_core.c:912:name=fips

As you can notice the above command fails to load the FIPS module. Does fipsinstall run successfully only on the machine that we built OpenSSL on?

We have a special use-case in which users would run our application on their machines, and in order to generate a FIPS configuration file on their machine, we bundle our application along with the openssl binary and the fips.so file. Is there some way to generate the FIPS configuration file, without bundling libgcc_s.so.0 object file too? We confirmed that copying libgcc_s.so.0 to the User Machine and adding its path to SHLIB_PATH helps run fipsinstall successfully.

Details on the User Machine

$ uname -a
HP-UX s-1537-v B.11.31 U ia64 3580688612 unlimited-user license

$ cc -V
(Bundled) cc: HP C/aC++ B3910B A.06.12 [Oct 11 2006]

$ which gcc
no gcc in /usr/local/bin /usr/sbin /usr/bin /usr/ccs/bin ...other directories in $PATH

Any help or pointers on this is greatly appreciated. Thanks!

[paulidale]

First problem: 3.0.7 isn't FIPS validated. You must build either 3.0.0 or 3.0.8 to get an approved FIPS provider.

Second problem: it looks like you need some kind of cross compilation setup. libgcc_s.so.0 isn't part of OpenSSL it comes with the compiler you're using and the dependency is being added when the .so is built. It might be possible to include this library inside the FIPS provider, this would require fiddling with the link line in a system dependent manner.

[sanchit-rubrik]

Hi @paulidale, thanks for the prompt reply.

We are aware that 3.0.7 is not FIPS validated. Just the major version (3.0.0) being FIPS validated works for us. We will sometime in the future think of upgrading to 3.0.8. But this issue that we are facing, I assume we'll also face this on other 3.0.0 versions, right?

Also, I tried building OpenSSL with -static-libgcc option. With this, I'm not facing this issue. Is using -static-libgcc a valid way to build OpenSSL?

Here are the exact commands used:

$ export CC=gcc; export CFLAGS="-pthread -O2 -g -mlp64 -D_GLIBCXX_USE_C99 -static-libgcc"; export CXXFLAGS="-pthread -O2 -g -mlp64 -D_GLIBCXX_USE_C99 -std=c++11"
$ ./Configure --prefix=openssl_build_dir --openssldir=openssl_build_dir hpux64-ia64-gcc no-shared enable-fips
$ gmake -j4
$ gmake install -j4