Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NAG 7.2 does not build 5.0.2, but does 4.1.6: -pthread passed to ld #12413

Open
mathomp4 opened this issue Mar 18, 2024 · 23 comments
Open

NAG 7.2 does not build 5.0.2, but does 4.1.6: -pthread passed to ld #12413

mathomp4 opened this issue Mar 18, 2024 · 23 comments

Comments

@mathomp4
Copy link

mathomp4 commented Mar 18, 2024

Background information

What version of Open MPI are you using? (e.g., v4.1.6, v5.0.1, git branch name and hash, etc.)

I'm testing Open MPI 5.0.2 (and 4.1.6)

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Both installed from tarfile.

Configure command is:

$ mkdir build-nag-7.2.01 && cd build-nag-7.2.01
$ ../configure --disable-wrapper-rpath --disable-wrapper-runpath \
      CC=gcc CXX=g++ FC=nagfor \
      --with-pmix=internal --with-hwloc=internal --with-libevent=internal \
      --prefix=/ford1/share/gmao_SIteam/MPI/openmpi/X.Y.Z/nag-7.2.01 |& tee configure.nag-7.2.01.log

where X.Y.Z is either 4.1.6 or 5.0.2

Please describe the system on which you are running

  • Operating system/version: RHEL8
  • Computer hardware: Intel(R) Xeon(R) CPU E5-2680 v4
  • Network type: Ethernet (I think? Single node)

Details of the problem

I am trying to build Open MPI 5.0.2 with NAG Fortran 7.2.01 (with GCC 12.1 as C/C++) and all works until:

make[3]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
  CCLD     libmpi_mpifh.la
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information

if I build with V=1:

make[3]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG  -finline-functions -mcx16 -version-info 80:0:40  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/5.0.2/n
ag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/ftmpi/mpif-h/libmpiext_ftmpi_mpifh.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/libopen-pal.la pr
ofile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la
 /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/libpmix.la
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/ftmpi/mpif-h/.libs/libmpiext_ftmpi_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/5.0.2/nag-7.2.01/lib -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs/libopen-pal.so -lpthread -lrt /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs/libpmix.so -lutil /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_core.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so -lm -ludev -ldl  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:2937: libmpi_mpifh.la] Error 1

So, I thought, let me try Open MPI 4.1.6 and:

...
make[3]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
  GENERATE sizeof_f.f90
  FC       sizeof_f.lo
NAG Fortran Compiler Release 7.2(Shin-Urayasu) Build 7201
[NAG Fortran Compiler normal termination]
  FCLD     libmpi_mpifh_sizeof.la
  CCLD     libmpi_mpifh.la
make[3]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[2]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
Making all in mpi/fortran/use-mpi-tkr
make[2]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/use-mpi-tkr'
  GENERATE mpi-tkr-sizeof.h
  GENERATE mpi-tkr-sizeof.f90
  FC       mpi_aint_add_f90.lo
  FC       mpi_aint_diff_f90.lo
  FC       mpi_comm_spawn_multiple_f90.lo
...

It builds successfully.

I looked around here and I did see #4032 which is old, but, well, NAG is there, pthread is there... Maybe some old fix for NAG and Open MPI 2 got into 3, stayed around for 4, but got lost with 5?

NOTE: I was going to try and maybe see if I could use our new NAG setup to help with #9795, but that is a configure time fail so...

@mathomp4
Copy link
Author

Oh, and I can say I've built Open MPI 5.0.2 with NAG 7.2 (and Apple Clang) on my macOS laptop, so I don't think is nagfor per se. Maybe something Clang is happy with but GCC isn't? 🤷🏼

@bosilca
Copy link
Member

bosilca commented Mar 18, 2024

gcc -dumpspecs | grep pthread and look for the lines starting with pthread. On my OSX with gcc (12.3.0 installed via macports) I see that -pthread will be translated at compile time into -D_REENTRANT

@mathomp4
Copy link
Author

mathomp4 commented Mar 18, 2024

@bosilca Well, on my mac, I build Open MPI for NAG with clang as the C compiler, not GCC:

❯ clang -dumpspecs | grep pthread
clang: error: unsupported option '-dumpspecs'
clang: error: no input files

But, on the RHEL 8 machine I see:

@$ gcc -dumpspecs | grep pthread
%{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT}
...

Of course, the error seems to say -pthread is not a flag ld knows (maybe via -Wl,-pthread from my verbose make) and:

@$ ld -pthread
ld: unrecognized option '-pthread'
ld: use the --help option for usage information

The verbose make says:

-ludev -ldl  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0

It's like configure has decided to pass -pthread to ld but that's not a flag ld knows?

@ggouaillardet
Copy link
Contributor

When nagfor is used, -pthread (that can be pulled from a .la file) is replaced with -Wl,-pthread via a patch we apply to config/ltmain.sh.

In your case, gcc is passed both -pthread and -Wl,-pthread which means ld will be passed the -pthread flag.

I do not quite see the connection between these ...

@ggouaillardet
Copy link
Contributor

since you built 4.1.6, can you please post the output of

cd ompi/mpi/fortran/mpif-h
rm libmpi_mpifh.la
make V=1

@mathomp4
Copy link
Author

@ggouaillardet Here is the make output for Open MPI 4.1.6:

Making all in profile
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -mcx16 -pthread -version-info 70:0:30  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/4.1.6/nag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/pcollreq/mpif-h/libmpiext_pcollreq_mpifh.la /ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/opal/libopen-pal.la profile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lrt -lm -lutil  -lz 
libtool: link: rm -fr  .libs/libmpi_mpifh.la .libs/libmpi_mpifh.lai .libs/libmpi_mpifh.so .libs/libmpi_mpifh.so.40 .libs/libmpi_mpifh.so.40.30.0 .libs/libmpi_mpifh.so.40.30.0T
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/pcollreq/mpif-h/.libs/libmpiext_pcollreq_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/orte/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/4.1.6/nag-7.2.01/lib -L/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/orte/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/opal/.libs ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/orte/.libs/libopen-rte.so /ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/opal/.libs/libopen-pal.so -ldl -ludev -lpciaccess -lrt -lm -lutil -lz  -O3 -mcx16 -pthread   -pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.30.0
libtool: link: (cd ".libs" && rm -f "libmpi_mpifh.so.40" && ln -s "libmpi_mpifh.so.40.30.0" "libmpi_mpifh.so.40")
libtool: link: (cd ".libs" && rm -f "libmpi_mpifh.so" && ln -s "libmpi_mpifh.so.40.30.0" "libmpi_mpifh.so")
libtool: link: ( cd ".libs" && rm -f "libmpi_mpifh.la" && ln -s "../libmpi_mpifh.la" "libmpi_mpifh.la" )
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'

and here is the same for 5.0.2:

Making all in profile
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG  -finline-functions -mcx16 -version-info 80:0:40  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/5.0.2/nag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/ftmpi/mpif-h/libmpiext_ftmpi_mpifh.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/libopen-pal.la profile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/libpmix.la
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/ftmpi/mpif-h/.libs/libmpiext_ftmpi_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/5.0.2/nag-7.2.01/lib -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs/libopen-pal.so -lpthread -lrt /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs/libpmix.so -lutil /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_core.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so -lm -ludev -ldl  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:2937: libmpi_mpifh.la] Error 1
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make: *** [Makefile:3435: all-recursive] Error 1

Doing a compare I see for 4.1.6:

-O3 -mcx16 -pthread   -pthread -Wl,-soname -Wl,libmpi_mpifh.so.40

and for 5.0.2:

-O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40

So a new -Wl, appeared (along with a lot of extra stuff of course).

@ggouaillardet
Copy link
Contributor

Thanks @mathomp4

would you mind giving a try with this 4.1.6 tarball?
https://github.com/ggouaillardet/ompi/releases/download/4.1.6/openmpi-4.1.6.tar.bz2

I regenerated a 4.1.6 tarball with the same autotools used by the v5 branch (they are slightly more recent that the autotools used in the v4.1 branch.

@mathomp4
Copy link
Author

@ggouaillardet Well, I'm not sure if it's a good thing or bad thing, but I tried your tarball and it built fine with NAG 7.2.01 on the same system.

@ggouaillardet
Copy link
Contributor

I concur, thanks for the test!

@ggouaillardet
Copy link
Contributor

I think I am on to something, but I do not quite see a good fix for that...

First the issue can be reproduced by

configure FC=nagfor && make
where nagfor can be a symbolic link to a working Fortran compiler (e.g. `gfortran)

with v4.1, from ompi/mpi/fortran/mpif-h/profile/Makefile:

LIBS = -lrt -lm -lutil  -lz  -lhwloc  -levent_core -levent_pthreads

but from v5.x, from `ompi/mpi/fortran/mpif-h/profile/Makefile:

LIBS = -lpthread -lrt -lm -lutil -levent_core -levent_pthreads /home/users/u0001043/build/openmpi-5.0.2-nagfor/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /home/users/u0001043/build/openmpi-5.0.2-nagfor/3rd-party/openpmix/src/libpmix.la

the major difference is v5 explicitly links with libpmix.la.
Long story short, libpmix.la depends on libhwloc.la and -Wl,-pthread gets pulled into the generated libraries, that will cause some issue later.

libpmix.la nor libhwloc.la are not needed here, so not using them might do the trick.

@jsquyres @bwbarrett any ideas on how to best move forward?

@mathomp4
Copy link
Author

mathomp4 commented Mar 26, 2024

This might be the first ever NAG bug I've had that doesn't require purchasing NAG! 😄

Thanks for working on this, @ggouaillardet

@ggouaillardet
Copy link
Contributor

@mathomp4 can you please give this commit a try?
ggouaillardet@8d331be

you will need to apply it on a v5 source tree and they run autogen.pl (or autogen.pl --force if applying on an official tarball)

Or you might simply download this already bootstrapped tarball at https://github.com/ggouaillardet/ompi/releases/download/nag_pmix/openmpi-gitclone.tar.bz2 and simply run configure ... && make

@mathomp4
Copy link
Author

mathomp4 commented Apr 1, 2024

@ggouaillardet I grabbed the tarball but:

...
NAG Fortran Compiler Release 7.2(Shin-Urayasu) Build 7201
[NAG Fortran Compiler normal termination]
  FCLD     libmpi_mpifh_sizeof.la
  CCLD     libmpi_mpifh.la
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:2938: libmpi_mpifh.la] Error 1
make[3]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[2]: *** [Makefile:3436: all-recursive] Error 1
make[2]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[1]: *** [Makefile:2801: all-recursive] Error 1
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi'
make: *** [Makefile:1534: all-recursive] Error 1

Should I try the source-and-autogen instead?

@ggouaillardet
Copy link
Contributor

I am puzzled since I could pass this with my nagfor symlink...

could you please make V=1 and share the output after CCLD libmpi_mpifh.la?

@mathomp4
Copy link
Author

mathomp4 commented Apr 1, 2024

Here is what I see:


make[3]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[3]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG  -finline-functions -mcx16 -version-info 80:0:40  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/5.0.2-gilles/nag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/ftmpi/mpif-h/libmpiext_ftmpi_mpifh.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/libopen-pal.la profile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/ftmpi/mpif-h/.libs/libmpiext_ftmpi_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/5.0.2-gilles/nag-7.2.01/lib ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/.libs/libopen-pal.so -ludev -ldl -lpthread -lrt -lm -lutil /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_core.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:2938: libmpi_mpifh.la] Error 1
make[3]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[2]: *** [Makefile:3436: all-recursive] Error 1
make[2]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[1]: *** [Makefile:2801: all-recursive] Error 1
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi'
make: *** [Makefile:1534: all-recursive] Error 1
mathomp4@gs6101-bucy .../FromGilles/openmpi-gitclone/build-nag-7.2.01

@ggouaillardet
Copy link
Contributor

can you please

cd ompi/mpi/fortran/mpif-h/profile
rm *.la
make V=1

-Wl,-pthread does not get pulled with my nagfor symlink (!)

@mathomp4
Copy link
Author

mathomp4 commented Apr 3, 2024

@ggouaillardet If I run those three steps, all is well.

But if, from my build dir I do:

cd ompi/mpi/fortran/mpif-h
rm *.la
make V=1

I get the fail:

@$ make V=1 |& tee make.log
Making all in profile
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=FC   --mode=link nagfor  -mismatch   -o libmpi_mpifh_sizeof.la  sizeof_f.lo  -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la
libtool: link: rm -fr  .libs/libmpi_mpifh_sizeof.a .libs/libmpi_mpifh_sizeof.la
libtool: link: ar cr .libs/libmpi_mpifh_sizeof.a .libs/sizeof_f.o
libtool: link: ranlib .libs/libmpi_mpifh_sizeof.a
libtool: link: ( cd ".libs" && rm -f "libmpi_mpifh_sizeof.la" && ln -s "../libmpi_mpifh_sizeof.la" "libmpi_mpifh_sizeof.la" )
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG  -finline-functions -mcx16 -version-info 80:0:40  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/5.0.2-gilles/nag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/ftmpi/mpif-h/libmpiext_ftmpi_mpifh.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/libopen-pal.la profile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/ftmpi/mpif-h/.libs/libmpiext_ftmpi_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/5.0.2-gilles/nag-7.2.01/lib ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/.libs/libopen-pal.so -ludev -ldl -lpthread -lrt -lm -lutil /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_core.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:2938: libmpi_mpifh.la] Error 1
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make: *** [Makefile:3436: all-recursive] Error 1
mathomp4@gs6101-bucy .../mpi/fortran/mpif-h

So it's like mpi/fortran/mpif-h/profile is good but not mpi/fortran/mpif-h

@ggouaillardet
Copy link
Contributor

Thanks. I expected the three steps to go well, but I wanted to see how profile/libmpi_mpifh_psizeof.la was built, so can you please share the output?

@mathomp4
Copy link
Author

mathomp4 commented Apr 4, 2024

Ahhh. My apologies. Here you go:

profile_make_log.txt

ETA: This is just complex enough I didn't want to paste. 😄

@ggouaillardet
Copy link
Contributor

Thanks, unfortunately I could not find anything relevant here ...

Instead of going back and forth, can you upload a tarball of your build directory? If too big, you can simply include all the config.log, Makefile and *.la files. I hope I will find where the -Wl,-pthread flags are pulled from.

@mathomp4
Copy link
Author

mathomp4 commented Apr 5, 2024

@ggouaillardet Here you go. This is my build dir but where I did --exclude=3rd-party to the tar command. Figured that was okay for a first try.

build-nag-7.2.01.tar.gz

@ggouaillardet
Copy link
Contributor

Thanks again!

So I finally found the issue. -pthread is pulled from the internal libevent library (I failed to see this because I was using my distro's libevent that did not provide the *.la files.

I uploaded a new tarball (same location) https://github.com/ggouaillardet/ompi/releases/download/nag_pmix/openmpi-gitclone.tar.bz2

could you please give it a try?

@mathomp4
Copy link
Author

mathomp4 commented Apr 8, 2024

@ggouaillardet I grabbed that tarfile and it configures, it builds, it installs, it checks, and Hello World says "Hello World".1

I think all is good!

Footnotes

  1. Well, use mpi says "Hello World". use mpi_f08 is still broken with NAG.

ggouaillardet added a commit to ggouaillardet/ompi that referenced this issue Apr 9, 2024
Fortran libraries do not need to be linked with PMIx, hwloc nor
libevent. That can cause some issues with incompatible flags are pulled
from the .la files.
For example, when the NAG Fortran compiler is used,
libmpi_mpifh_sizeof.la pulls the -pthread flag, that is translated
into -Wl,-pthread.
Then when libmpi_mpifh.la is linked with gcc but pulls -Wl,-pthread that
causes the build to fail since -pthread is not a valid ld option.
Not depending on these libraries is enough to avoid the above issue.

Refs open-mpi#12413

Thanks to Matthew Thompson for reporting this issue and helping with
the diagnostic.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
ggouaillardet added a commit to ggouaillardet/ompi that referenced this issue Apr 10, 2024
Fortran libraries do not need to be linked with PMIx, hwloc nor
libevent. That can cause some issues with incompatible flags are pulled
from the .la files.
For example, when the NAG Fortran compiler is used,
libmpi_mpifh_sizeof.la pulls the -pthread flag, that is translated
into -Wl,-pthread.
Then when libmpi_mpifh.la is linked with gcc but pulls -Wl,-pthread that
causes the build to fail since -pthread is not a valid ld option.
Not depending on these libraries is enough to avoid the above issue.

Refs open-mpi#12413

Thanks to Matthew Thompson for reporting this issue and helping with
the diagnostic.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants