New cctools still broken on osx-arm64

[erykoff]

• I read the conda-forge documentation and could not find the solution for my problem there.

Issue:

this issue conda-forge/admin-requests#324 is still a problem. New builds came out and lead to unsigned binaries on osx-arm64. I don’t know the underlying cause or how to keep us from releasing updates on this.
@isuruf
@beckermr

Environment (conda list):

$ conda list

Details about conda and system ( conda info ):

$ conda info

[isuruf]

Can you give a MWE?

[rgommers]

Here is a reproducer (from rgommers/scipy#134) on an arm64 machine:

# clone https://github.com/scipy/scipy and cd to root of repo
% conda env create -f environment_meson.yml
% conda activate scipy-meson
% python dev.py --build-only
...
meson.build:41:0: ERROR: Executables created by Fortran compiler /Users/sayantikabanik/miniforge3/envs/scipy-dev/bin/arm64-apple-darwin20.0.0-gfortran are not runnable.

Downgrading ld64 from build number 609 to 530 fixes the problem.

Given that it's in the configure stage of the build, there's nothing scipy-specific going on there - it looks like any use of gfortran will have that issue.

[isuruf]

Would you be able to reduce this down to a small Fortran/C only example so we can report it upstream?

[rgommers]

Yes I think so, hopefully later today (first a bunch of meetings to go to).

[rgommers]

The problem seemed to be an absolute path given to -o. With a test file sanitycheckf.f90:

program main; print *, "Fortran compilation is working."; end program

Adding some debug prints shows what arguments are used when trying to configure the SciPy build:

['/Users/rgommers/mambaforge/envs/scipy-meson-arm64/bin/arm64-apple-darwin20.0.0-gfortran', '-Wno-conversion', '-Wl,-pie', '-Wl,-headerpad_max_install_names', '-Wl,-dead_strip_dylibs', '-Wl,-rpath,/Users/rgommers/mambaforge/envs/scipy-meson-arm64/lib', '-L/Users/rgommers/mambaforge/envs/scipy-meson-arm64/lib', '/Users/rgommers/code/scipy/build/meson-private/sanitycheckf.f90', '-o', '/Users/rgommers/code/scipy/build/meson-private/sanitycheckf']

['/Users/rgommers/code/scipy/build/meson-private/sanitycheckf']
-9

meson.build:41:0: ERROR: Executables created by Fortran compiler /Users/rgommers/mambaforge/envs/scipy-meson-arm64/bin/arm64-apple-darwin20.0.0-gfortran are not runnable.

So running that manually, putting the test binary in the root of the repo, with a relative path (-o sanitycheck):

% /Users/rgommers/mambaforge/envs/scipy-meson-arm64/bin/arm64-apple-darwin20.0.0-gfortran -Wno-conversion -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/rgommers/mambaforge/envs/scipy-meson-arm64/lib -L/Users/rgommers/mambaforge/envs/scipy-meson-arm64/lib /Users/rgommers/code/scipy/build/meson-private/sanitycheckf.f90 -o sanitycheckf
% ./sanitycheckf 
 Fortran compilation is working.

Now with an absolute path:

% /Users/rgommers/mambaforge/envs/scipy-meson-arm64/bin/arm64-apple-darwin20.0.0-gfortran -Wno-conversion -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/rgommers/mambaforge/envs/scipy-meson-arm64/lib -L/Users/rgommers/mambaforge/envs/scipy-meson-arm64/lib /Users/rgommers/code/scipy/build/meson-private/sanitycheckf.f90 -o /Users/rgommers/code/scipy/sanitycheckf
% ./sanitycheckf 
zsh: killed     ./sanitycheckf

Stripping that further down:

% rm sanitycheckf
% gfortran build/meson-private/sanitycheckf.f90 -o /Users/rgommers/code/scipy/sanitycheckf       
% ./sanitycheckf 
zsh: killed     ./sanitycheckf
% rm sanitycheckf
% gfortran build/meson-private/sanitycheckf.f90 -o sanitycheckf 
% ./sanitycheckf                                              
 Fortran compilation is working.

Showing that a simple absolute path is enough even with no other compiler arguments.

[beckermr]

Wow. That is a great bug report! Thank you!

[isuruf]

Thanks. Can you open it at https://github.com/tpoechtrager/cctools-port ?

[isuruf]

I assume following works for you as well right?

gfortran build/meson-private/sanitycheckf.f90 -o /Users/rgommers/code/scipy/sanitycheckf       
cp ./sanitycheckf sanitycheckf2
cp sanitycheckf2 sanitycheckf
./sanitycheckf

[erykoff]

FYI, I just checked and it's not just gfortran, it's clang (no surprise). Great find on the abs path issue, @rgommers !

And strangely, @isuruf is correct that copying fixes it ... so presumably you know what's going wrong?

$ clang /Users/erykoff/src/hello_world/hello_world.c -o /Users/erykoff/src/hello_world/hello_world
$ ./hello_world 
zsh: killed     ./hello_world
$ cp hello_world hello_world2
$ cp hello_world2 hello_world
$ ./hello_world
Hello, world!

[rgommers]

Thanks. Can you open it at https://github.com/tpoechtrager/cctools-port ?

Done in tpoechtrager/cctools-port#117