Cannot import gpboost on an M1 Mac

[ahalev]

I am having an issue importing gpboost on an M1 Mac (Silicon Chip).

When I call

import gpboost as gpb

I get the following exception:

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/__init__.py", line 6, in <module>
    from .basic import Booster, Dataset, register_logger, GPModel
  File "/opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/basic.py", line 101, in <module>
    _LIB = _load_lib()
  File "/opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/basic.py", line 92, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "/opt/anaconda3/envs/swine/lib/python3.7/ctypes/__init__.py", line 442, in LoadLibrary
    return self._dlltype(name)
  File "/opt/anaconda3/envs/swine/lib/python3.7/ctypes/__init__.py", line 364, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/lib_gpboost.so, 0x0006): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
  Referenced from: /opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/lib_gpboost.so
  Reason: tried: '/usr/local/opt/libomp/lib/libomp.dylib' (no such file), '/usr/local/lib/libomp.dylib' (no such file), '/usr/lib/libomp.dylib' (no such file)
python-BaseException

which suggests that none of these files exist. However, after installing libomp from brew, this runs fine:

(base) MacBook-Pro-6:lib ahalev$ ls -l /usr/local/lib/libomp.dylib
lrwxr-xr-x  1 ahalev  admin  30 Apr 29 01:44 /usr/local/lib/libomp.dylib -> /opt/homebrew/lib/libomp.dylib

Any suggestions would be appreciated. Perhaps this dependency be removed, as was done in PyTorch here?

[fabsig]

@ahalev: Thanks a lot for reporting this.

So does brew install libomp solve the problem? As far as I understand it, the dependency on libomp cannot be removed.

[ahalev]

brew install libomp doesn't solve the issue, but changes the exception to

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/__init__.py", line 6, in <module>
    from .basic import Booster, Dataset, register_logger, GPModel
  File "/opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/basic.py", line 101, in <module>
    _LIB = _load_lib()
  File "/opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/basic.py", line 92, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "/opt/anaconda3/envs/swine/lib/python3.7/ctypes/__init__.py", line 442, in LoadLibrary
    return self._dlltype(name)
  File "/opt/anaconda3/envs/swine/lib/python3.7/ctypes/__init__.py", line 364, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/lib_gpboost.so, 0x0006): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
  Referenced from: /opt/anaconda3/envs/swine/lib/python3.7/site-packages/gpboost/lib_gpboost.so
  Reason: tried: '/usr/local/opt/libomp/lib/libomp.dylib' (no such file), '/usr/local/lib/libomp.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/lib/libomp.dylib' (no such file)
python-BaseException

Note the reason change to '/usr/local/lib/libomp.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')) in the final line.

Installing with arch -x86_64 /usr/local/homebrew/bin/brew install libomp, which presumably is the correct architecture, does not change anything here.

[fabsig]

@ahalev : Have you tried the installation with gcc instead of clang running these steps first?

[fabsig]

Closing. See #51 for updates.

[fabsig]

A macOS arm64 chip pre-compiled version is now on PyPI. You should now be able to install with pip on M1 / M2 Macs without any problems as compilation is not required anymore.