MacOS install error: Library not loaded: @rpath/libc++.1.dylib

[koaning]

🐛 Bug

Install on MacOS fails with pip.

To Reproduce

(venv) ➜  whatlies git:(master) ✗ python -m pip install torch
(venv) ➜  whatlies git:(master) ✗ python                            
Python 3.7.7 (v3.7.7:d7c567b08f, Mar 10 2020, 02:56:16) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/vincent/Development/whatlies/venv/lib/python3.7/site-packages/torch/__init__.py", line 81, in <module>
    from torch._C import *
ImportError: dlopen(/Users/vincent/Development/whatlies/venv/lib/python3.7/site-packages/torch/_C.cpython-37m-darwin.so, 9): Library not loaded: @rpath/libc++.1.dylib
  Referenced from: /Users/vincent/Development/whatlies/venv/lib/python3.7/site-packages/torch/_C.cpython-37m-darwin.so
  Reason: image not found

Google suggested doing this;

(venv) ➜  whatlies git:(master) ✗ brew install libomp               

That unfortunately did not work.

Expected behavior

No error.

Environment

(venv) ➜  whatlies git:(master) ✗ python collect_env.py 
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A

OS: Mac OSX 10.15.3
GCC version: Could not collect
CMake version: Could not collect

Python version: 3.7
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.18.3
[pip3] torch==1.3.1
[pip3] torchvision==0.5.0
[conda] Could not collect

cc @ezyang @gchanan @zou3519 @bdhirsh @seemethere @malfet @walterddr

[koaning]

Note that this error also occurs on 1.4.0.

(venv) ➜  whatlies git:(master) ✗ python collect_env.py      
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A

OS: Mac OSX 10.15.3
GCC version: Could not collect
CMake version: Could not collect

Python version: 3.7
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.18.3
[pip3] torch==1.4.0
[pip3] torchvision==0.5.0
[conda] Could not collect
(venv) ➜  whatlies git:(master) ✗ import python
zsh: command not found: import
(venv) ➜  whatlies git:(master) ✗ python
Python 3.7.7 (v3.7.7:d7c567b08f, Mar 10 2020, 02:56:16) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/vincent/Development/whatlies/venv/lib/python3.7/site-packages/torch/__init__.py", line 81, in <module>
    from torch._C import *
ImportError: dlopen(/Users/vincent/Development/whatlies/venv/lib/python3.7/site-packages/torch/_C.cpython-37m-darwin.so, 9): Library not loaded: @rpath/libc++.1.dylib
  Referenced from: /Users/vincent/Development/whatlies/venv/lib/python3.7/site-packages/torch/_C.cpython-37m-darwin.so
  Reason: image not found

[ezyang]

Does conda work for you?

[soundmartell]

lib/python3.7/site-packages/torch/init.py", line 82, in
from torch._C import *

[koaning]

@ezyang I typically don't use conda and I haven't tried it. The docs suggest it is not needed though.

[soundmartell]

Ok, I solved the issue, I don't know if it has any unwanted impact in any way but now Torch is working.
Please, note I am using Mac Pycharm with a virtual environment so...
I copied: libc++.1.dylib from usr/lib
and I pace it in my environment: venv/lib/python3.7/site-packages/torch/lib

I tested Torch with this script:

from future import print_function
import torch
x = torch.rand(5, 3)
print(x)

It devolved:

tensor([[0.3633, 0.7173, 0.6055],
[0.3442, 0.6892, 0.2950],
[0.3909, 0.4718, 0.0202],
[0.1055, 0.8912, 0.0667],
[0.3819, 0.4413, 0.1544]])

I hope it work for others. I have been 3 days trying to find solutions in Google but not any solution found it.

[kaiwencui]

Ok, I solved the issue, I don't know if it has any unwanted impact in any way but now Torch is working.
Please, note I am using Mac Pycharm with a virtual environment so...
I copied: libc++.1.dylib from usr/lib
and I pace it in my environment: venv/lib/python3.7/site-packages/torch/lib

I tested Torch with this script:

from future import print_function
import torch
x = torch.rand(5, 3)
print(x)

It devolved:

tensor([[0.3633, 0.7173, 0.6055],
[0.3442, 0.6892, 0.2950],
[0.3909, 0.4718, 0.0202],
[0.1055, 0.8912, 0.0667],
[0.3819, 0.4413, 0.1544]])

I hope it work for others. I have been 3 days trying to find solutions in Google but not any solution found it.

Can confirm this method works on macOS running on Python 3.8 in venv

[koaning]

Running this works on my side too.

 cp /usr/lib/libc++.1.dylib venv/lib/python3.7/site-packages/torch/lib 

Seems like something is not properly packaged along?

[ezyang]

Yes, it sounds like the rpath for libc++ is incorrectly set on the wheel packages. We should not be rpath'ing it at all.

[mattip]

maybe connected to gh-35418, where rpaths to cuda libraries are suspected of causing problems

[puthurr]

Hi All

Have the same issue with torchtext though (torch alone is fine)

Python 3.7.7 (v3.7.7:d7c567b08f, Mar 10 2020, 02:56:16)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import torchtext
Traceback (most recent call last):
File "", line 1, in
File "/Users/nicolas/projects/PyTorch/venv/lib/python3.7/site-packages/torchtext/init.py", line 42, in
_init_extension()
File "/Users/nicolas/projects/PyTorch/venv/lib/python3.7/site-packages/torchtext/init.py", line 38, in _init_extension
torch.ops.load_library(ext_specs.origin)
File "/Users/nicolas/projects/PyTorch/venv/lib/python3.7/site-packages/torch/_ops.py", line 105, in load_library
ctypes.CDLL(path)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ctypes/init.py", line 364, in init
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/nicolas/projects/PyTorch/venv/lib/python3.7/site-packages/torchtext/_torchtext.so, 6): Library not loaded: @rpath/libc++.1.dylib
Referenced from: /Users/nicolas/projects/PyTorch/venv/lib/python3.7/site-packages/torchtext/_torchtext.so
Reason: image not found

Tried the workaround but didn't work.

LDD equivalent shows the following
_otool -L /Users/nicolas/projects/PyTorch/venv/lib/python3.7/site-packages/torchtext/_torchtext.so
/Users/nicolas/projects/PyTorch/venv/lib/python3.7/site-packages/torchtext/torchtext.so:
@rpath/libc10.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libtorch.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libtorch_cpu.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libtorch_python.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)

[malfet]

@puthurr what versions of PyTorch and TorchText are you trying to install?

[puthurr]

Just installing the latest available
torch in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (1.6.0)
torchvision in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (0.7.0)
torchtext in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (0.7.0)

[wenqinglim]

@puthurr I ran the solution suggested above cp /usr/lib/libc++.1.dylib venv/lib/python3.7/site-packages/torch/lib and also still faced the same issue as you described. My Pytorch was also working, but TorchText was not. I was using:

Python 3.7.2
torchtext==0.7.0
torch==1.4.0

I managed to resolve it by upgrading to torch==1.6.0.
I see above that you're already on Torch 1.6.0, but you might want to do a double check on your other library versions and their compatibility.

[puthurr]

@malfet @wenqinglim
I solved it by changing the rpath in the library _torchtext.so using the following command (Mac OS X)
install_name_tool -change @rpath/libc++.1.dylib /usr/lib/libc++.1.dylib _torchtext.so
Now the path is absolute.
otool -L _torchtext.so
_torchtext.so:
@rpath/libc10.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libtorch.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libtorch_cpu.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libtorch_python.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)

thus importing torchtext works fine now. Not ideal but working.

[jqiu2]

I have the same issue with torch audio, I ran the command above and face the same issues.

[PearlW1]

brew install libomp

Don't run "brew install libomp", it destroyed my environment , then i create a new environment, install other configuration。

[jrmh96]

I am also having trouble with some of the dependencies on Mac Big Sur, specifically
from pytorch3d import _C

yields

Referenced from: venv/lib/python3.8/site-packages/pytorch3d/_C.cpython-38-darwin.so Reason: Incompatible library version: _C.cpython-38-darwin.so requires version 1.0.0 or later, but libc10.dylib provides version 0.0.0

This is after changing the rpath using install_name_tool as @puthurr suggested

Environment:
torch==1.9.0
pytorch3d==1.3.0

[espoirMur]

@puthurr I ran the solution suggested above cp /usr/lib/libc++.1.dylib venv/lib/python3.7/site-packages/torch/lib and also still faced the same issue as you described. My Pytorch was also working, but TorchText was not. I was using:

Python 3.7.2
torchtext==0.7.0
torch==1.4.0

I managed to resolve it by upgrading to torch==1.6.0.
I see above that you're already on Torch 1.6.0, but you might want to do a double check on your other library versions and their compatibility.

upgrading to torch 1.6.0 solved the issue for me.

[jskye]

i have basically the same problem but the library libshm.dylib is missing.

dlopen(/usr/local/lib/python3.9/site-packages/torch/_C.cpython-39-darwin.so, 2): Library not loaded: @loader_path/libshm.dylib
  Referenced from: /usr/local/lib/python3.9/site-packages/torch/lib/libtorch_python.dylib
  Reason: image not found

all answers i found to fix that say to install libomp, which i reinstalled but it didnt fix the issue, though for others that seemed to work in previous years.

i cant find the library libshm in any lib folders on my machine. Im wondering if its included somewhere in libomp and how to point pytorch to it, or if it can be downloaded from somewhere else?

i have the current torch version installed 1.11.0 with Python 3.9 on Mac Big Sur.

Found the libshm library required for sharedmemory alloc on linux machines.
Does PyTorch require a bugfix to include this dylib? Or is there any other workaround? @malfet

[malfet]

@jskye Can you please run collect_env and post results here? (Or better open a separate issue)

[jskye]

I received this error when installing DALLE2.
i think the problem was that the dream lib it relies on big sleep library which requires CUDA that i dont have as my mac has an older onboard Intel chip. big sleep uses CUDA..
I think the issue pertaining to the above libshm may have been due to some mismatch as i think i had pytorch version 12 installed possibly from anaconda and i also had torchaudio and torchvideo that required the older version 11.
In any case after resolving that mismatch i get this CUDA error again.
Indeed some other experimental art AIs on google collab required me to switch over to use the CPU to use.

dream 'sharing a sunset at the summit of mount everest with my dog'
Traceback (most recent call last):
  File "/usr/local/bin/dream", line 5, in <module>
    from big_sleep.cli import main
  File "/usr/local/lib/python3.9/site-packages/big_sleep/__init__.py", line 1, in <module>
    from big_sleep.big_sleep import BigSleep, Imagine
  File "/usr/local/lib/python3.9/site-packages/big_sleep/big_sleep.py", line 26, in <module>
    assert torch.cuda.is_available(), 'CUDA must be available in order to use Big Sleep'