Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] FEA Support Veclib/Accelerate #166

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

jeremiedbb
Copy link
Collaborator

@jeremiedbb jeremiedbb commented Feb 9, 2024

Fixes #135.

@jeremiedbb
Copy link
Collaborator Author

I don't think it's possible to support Accelerate:

  • when installed by conda-forge, there's only a libblas.dylib (whereas openblas for instance comes with libblas.dylib and libopenblas.dylib) so threadpoolctl can't rely on the lib name only to guess which one it is.
  • this libblas.dylib seems to not expose any symbol that allows to identify accelerate. It only exposes the generic blas symbols. I say that based on running nm -a libblas.dylib, where the only non generic symbol _veclib which is a local (non-external) bss (block starting symbol) section symbol which is not accessible in python using hasattr.

@ogrisel
Copy link
Contributor

ogrisel commented Feb 9, 2024

When using otool I can see that it's actually linked to libvecLibFort-ng via an @rpath entry:

otool -L /Users/ogrisel/mambaforge-disabled/envs/dev/lib/libblas.3.dylib
/Users/ogrisel/mambaforge-disabled/envs/dev/lib/libblas.3.dylib:
        @rpath/libvecLibFort-ng.dylib (compatibility version 0.0.0, current version 0.0.0)
        /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib (compatibility version 1.0.0, current version 1.0.0, reexport)
        @rpath/liblapack-netlib.3.9.0.dylib (compatibility version 0.0.0, current version 0.0.0, reexport)
        @rpath/liblapacke-netlib.3.9.0.dylib (compatibility version 0.0.0, current version 0.0.0, reexport)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)

@ogrisel
Copy link
Contributor

ogrisel commented Feb 13, 2024

Here are known cases of numpy being linked accelerate in the wild:

Note that the various packaging methods my differ in how vecLib is linked to the Python program. This might introduce additional difficulty in finding a robust heuristic to detect accelerate/vecLib given the fact that there are no vecLib-specific exported symbols, as far as I know.

@jeremiedbb
Copy link
Collaborator Author

So irl we discussed the possibility to check if the detected libblas.dylib has a dependency over libvecLibFort-ng.dylib, but I didn't manage to do that easily.

Although on linux it's seems doable using dlinfo, this is gnu only, and I didn't an equivalent for macos. The dyld functions seems too limited (https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/dyld.3.html).

I tried to look at the source code of otool to find how they do it but the solution seems way to complicated...

I have an alternative that I don't particularly like. When we find a libblas we create a subprocess where we only dlopen this libblas and check if _find_libraries_with_dyld finds libvecLibFort-ng.dylib. It would not add that much complexity to the code base but the cost of creating a new process is clearly not negligible.

@jeremiedbb
Copy link
Collaborator Author

Another alternative that we discussed irl but I'm not a fan either is to directly check for libvecLibFort-ng.dylib but it's not the real dylib containing the blas, and the info dict would not show the path to libblas either.

@jeremiedbb
Copy link
Collaborator Author

Anyway, let's not delay the release any longer for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Detect the Apple Accelerate/vecLib runtime
2 participants