I can't import PyTorch, libomp.dylib can't be loaded. #20030

kaszperro · 2019-05-01T21:19:45Z

🐛 Bug

I tried to install PyTorch, but I can't use it. It can't link dynamic library libomp.dylib

To Reproduce

Steps to reproduce the behavior:

pipenv install torch torchvision
from torch.utils.data import Dataset

Traceback (most recent call last):
  File "/Users/kaspersapala/Documents/Airly/new-pollution-forecast/Datasets/__init__.py", line 3, in <module>
    from torch.utils.data import Dataset
  File "/Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/__init__.py", line 79, in <module>
    from torch._C import *
ImportError: dlopen(/Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/_C.cpython-37m-darwin.so, 9): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
  Referenced from: /Users/kaspersapala/.local/share/virtualenvs/new-pollution-forecast-BSetKF9E/lib/python3.7/site-packages/torch/lib/libshm.dylib
  Reason: image not found

Expected behavior

Being able to use and import PyTorch

Environment

PyTorch Version (e.g., 1.0): 1.1.0
OS (e.g., Linux): Mac OS Mojave
How you installed PyTorch (conda, pip, source): pipenv
Python version: 3.7.1

The text was updated successfully, but these errors were encountered:

kaszperro · 2019-05-01T21:49:29Z

brew install libomp solves the problem.

pytorch/pytorch#20030

* Fix the error when importing torch (1.1.0) on osx pytorch/pytorch#20030 * Update torch version to 1.1.0 for linux * Update the doc for OS X about an external dependency on libomp

ezyang · 2019-05-06T17:16:03Z

cc @JerryShih

ezyang · 2019-05-06T17:17:00Z

cc @pjh5 @kostmo

JerryShih · 2019-05-07T08:45:11Z

brew install libomp solves the problem.

@ezyang
How to add the libomp dependency in pytorch pip installation?

Will all pytorch dependent libraries be backed into wheel package?
If yes, I will add the libomp files like these code:

pytorch/setup.py

Lines 738 to 825 in 2356fac

    
           package_data={ 
        
               'torch': [ 
        
                   'py.typed', 
        
                   'bin/*', 
        
                   'test/*', 
        
                   '__init__.pyi', 
        
                   'cuda/*.pyi', 
        
                   'optim/*.pyi', 
        
                   'autograd/*.pyi', 
        
                   'utils/data/*.pyi', 
        
                   'lib/*.so*', 
        
                   'lib/*.dylib*', 
        
                   'lib/*.dll', 
        
                   'lib/*.lib', 
        
                   'lib/*.pdb', 
        
                   'lib/torch_shm_manager', 
        
                   'lib/*.h', 
        
                   'include/ATen/*.h', 
        
                   'include/ATen/cpu/*.h', 
        
                   'include/ATen/cpu/vec256/*.h', 
        
                   'include/ATen/core/*.h', 
        
                   'include/ATen/cuda/*.cuh', 
        
                   'include/ATen/cuda/*.h', 
        
                   'include/ATen/cuda/detail/*.cuh', 
        
                   'include/ATen/cuda/detail/*.h', 
        
                   'include/ATen/cudnn/*.h', 
        
                   'include/ATen/detail/*.h', 
        
                   'include/caffe2/utils/*.h', 
        
                   'include/c10/*.h', 
        
                   'include/c10/macros/*.h', 
        
                   'include/c10/core/*.h', 
        
                   'include/ATen/core/dispatch/*.h', 
        
                   'include/ATen/core/op_registration/*.h', 
        
                   'include/c10/core/impl/*.h', 
        
                   'include/c10/util/*.h', 
        
                   'include/c10/cuda/*.h', 
        
                   'include/c10/cuda/impl/*.h', 
        
                   'include/c10/hip/*.h', 
        
                   'include/c10/hip/impl/*.h', 
        
                   'include/caffe2/**/*.h', 
        
                   'include/torch/*.h', 
        
                   'include/torch/csrc/*.h', 
        
                   'include/torch/csrc/api/include/torch/*.h', 
        
                   'include/torch/csrc/api/include/torch/data/*.h', 
        
                   'include/torch/csrc/api/include/torch/data/dataloader/*.h', 
        
                   'include/torch/csrc/api/include/torch/data/datasets/*.h', 
        
                   'include/torch/csrc/api/include/torch/data/detail/*.h', 
        
                   'include/torch/csrc/api/include/torch/data/samplers/*.h', 
        
                   'include/torch/csrc/api/include/torch/data/transforms/*.h', 
        
                   'include/torch/csrc/api/include/torch/detail/*.h', 
        
                   'include/torch/csrc/api/include/torch/detail/ordered_dict.h', 
        
                   'include/torch/csrc/api/include/torch/nn/*.h', 
        
                   'include/torch/csrc/api/include/torch/nn/modules/*.h', 
        
                   'include/torch/csrc/api/include/torch/nn/parallel/*.h', 
        
                   'include/torch/csrc/api/include/torch/optim/*.h', 
        
                   'include/torch/csrc/api/include/torch/serialize/*.h', 
        
                   'include/torch/csrc/autograd/*.h', 
        
                   'include/torch/csrc/autograd/functions/*.h', 
        
                   'include/torch/csrc/autograd/generated/*.h', 
        
                   'include/torch/csrc/autograd/utils/*.h', 
        
                   'include/torch/csrc/cuda/*.h', 
        
                   'include/torch/csrc/jit/*.h', 
        
                   'include/torch/csrc/jit/generated/*.h', 
        
                   'include/torch/csrc/jit/passes/*.h', 
        
                   'include/torch/csrc/jit/passes/utils/*.h', 
        
                   'include/torch/csrc/jit/script/*.h', 
        
                   'include/torch/csrc/jit/testing/*.h', 
        
                   'include/torch/csrc/onnx/*.h', 
        
                   'include/torch/csrc/utils/*.h', 
        
                   'include/pybind11/*.h', 
        
                   'include/pybind11/detail/*.h', 
        
                   'include/TH/*.h*', 
        
                   'include/TH/generic/*.h*', 
        
                   'include/THC/*.cuh', 
        
                   'include/THC/*.h*', 
        
                   'include/THC/generic/*.h', 
        
                   'include/THCUNN/*.cuh', 
        
                   'include/THCUNN/generic/*.h', 
        
                   'include/THNN/*.h', 
        
                   'include/THNN/generic/*.h', 
        
                   'share/cmake/ATen/*.cmake', 
        
                   'share/cmake/Caffe2/*.cmake', 
        
                   'share/cmake/Caffe2/public/*.cmake', 
        
                   'share/cmake/Caffe2/Modules_CUDA_fix/*.cmake', 
        
                   'share/cmake/Caffe2/Modules_CUDA_fix/upstream/*.cmake', 
        
                   'share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindCUDA/*.cmake', 
        
                   'share/cmake/Gloo/*.cmake', 
        
                   'share/cmake/Torch/*.cmake',

How about other dependent libraries? Are they all in third_party folder and built from source?

ezyang · 2019-05-07T13:27:48Z

pip doesn't know anything about brew, you can't ask it to brew install a package. Also, while this resolution "solves" the issue, I would love to better understand how this dependency came to be (in particular, why are we building binaries with a dependency on brew libraries? That sounds bad.)

In some cases where we can't assume end users have a library installed, we distribute the library with our binaries. However, I'm not sure that we should do this for libomp.dylib. cc @soumith

soumith · 2019-05-07T15:11:02Z

this is something that needs to be fixed. I can take a look at fixing binaries and re-uploading.
The problem is that our macos CI installs libomp over here:

pytorch/.circleci/verbatim-sources/header-section.yml

Line 294 in 0da0c4b

brew install libomp

, I think for CI / testing. But binary builds also pick it up and use it.

pjh5 · 2019-05-07T16:17:20Z

I can remove this dependency for the binaries. I'll put out a PR soon

dudongge · 2019-05-13T08:31:00Z

brew install libomp solves the proble
this solve my problem

gongenbo · 2019-05-31T06:45:13Z

brew install libomp solves the problem.

It work!

adrienchaton · 2019-06-05T13:17:40Z

when trying to upgrade pytorch from 1.0.1 to 1.1, I ran into the same issue and found this thread

the update on linux server worked fine but not on my macbook (OSX)

I tried brew install libomp but the brew formula could not be found, hence this did not fix the issue and I had to downgrade back to 1.0.1 on OSX to be able to import torch again

did anyone have the same issue regarding both the pip install of pytorch and the brew install of libomp ?

akirchhoff-modular · 2019-06-05T21:01:34Z

@soumith

I can take a look at fixing binaries and re-uploading.

Is there still any plan to do this, or are new binary releases going to wait for the next PyTorch release?

kaszperro · 2019-06-05T21:09:02Z

when trying to upgrade pytorch from 1.0.1 to 1.1, I ran into the same issue and found this thread

the update on linux server worked fine but not on my macbook (OSX)

I tried brew install libomp but the brew formula could not be found, hence this did not fix the issue and I had to downgrade back to 1.0.1 on OSX to be able to import torch again

did anyone have the same issue regarding both the pip install of pytorch and the brew install of libomp ?

You need to install brew first, go to brew site and follow install instructions.

adrienchaton · 2019-06-06T12:05:42Z

thanks for your reply, I had brew installed but it was not enough up to date
so I updated everything and now it works

note for other users, when I did brew install libomp, it went through but other applications did not work anymore (Spyder IDE) ; at this moment my OS was still 10.11 (Capitan) and I had to upgrade to 10.12 (Sierra) to be able to solve the problem

so from my experience, I would recommend upgrading to Sierra at least before installing pytorch 1.1 and libomp (I use pip and brew for all install)

soumith · 2019-06-21T22:10:51Z

uploaded new binaries that remove dependency on libomp.dylib

sukuya · 2019-07-11T03:27:03Z

Now I get clang: error: unsupported option '-fopenmp' error.

akirchhoff-modular · 2019-07-11T17:27:28Z

@sukuya That sounds like a different problem, since it sounds like you're trying to compile PyTorch yourself, but this issue concerned a problem with the pre-compiled binaries.

torrmal · 2019-07-13T00:55:33Z

this issue still present in torch 1.1.0.post2, do you know when the fix will be available on stable release?

akirchhoff-modular · 2019-07-13T00:59:25Z

@torrmal How are you making sure that you are using 1.1.0.post2? In my testing the changes introduced by 1.1.0.post2 fixed this problem completely, so I wonder whether you have a 1.1.0 non-post2 lying around somewhere that's getting picked up inadvertently.

sukuya · 2019-07-13T02:13:12Z

@sukuya That sounds like a different problem, since it sounds like you're trying to compile PyTorch yourself, but this issue concerned a problem with the pre-compiled binaries.

@alex-xnor I was compiling a c++ program which used prebuilt libtorch and I get those errors.

pjh5 · 2019-07-15T16:07:17Z

@sukuya can you make a new issue with more details on your problem? In particular, can you include the cmake/makefile that you are running and the compiler command that you are using?

GPPyeye · 2020-05-12T06:22:55Z

brew install libomp solves the problem.

Thx bro I've solved my problem

leedrake5 · 2020-06-20T04:25:06Z

Does anyone have a suggestion for when brew install libomp doesn't work? I've reinstalled, relinked, but to no avail.

MattioCh · 2020-07-08T12:04:54Z

put this in terminal,hope it works. It took me 2hrs to fix this and I hope you can fix it faster, jeeez...

install_name_tool -change @rpath/libc++.1.dylib /usr/lib/libc++.1.dylib /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_cvxcore.cpython-37m-darwin.so

PearlW1 · 2020-12-22T03:58:31Z

brew install libomp solves the problem.

Don't run brew install libomp, it destroyed my environment, then i create a new env, install other configuration, so sad.

callmejliu · 2023-04-03T00:23:35Z

I also have the same issue when Launching Web UI with arguments:... Anyone have the same issue being solved?
I tried v1.5, 2.0 and 2.1 also have the .yaml file installed in the same folder.

Installing requirements for Web UI
Launching Web UI with arguments: --opt-sub-quad-attention --skip-torch-cuda-test --no-half-vae --use-cpu interrogate
Traceback (most recent call last):
File "/Users/jeffreyhomeimac/stable-diffusion-webui/launch.py", line 356, in
start()
File "/Users/jeffreyhomeimac/stable-diffusion-webui/launch.py", line 347, in start
import webui
File "/Users/jeffreyhomeimac/stable-diffusion-webui/webui.py", line 16, in
from modules import paths, timer, import_hook, errors
File "/Users/jeffreyhomeimac/stable-diffusion-webui/modules/paths.py", line 5, in
import modules.safe
File "/Users/jeffreyhomeimac/stable-diffusion-webui/modules/safe.py", line 9, in
import torch
File "/Users/jeffreyhomeimac/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/init.py", line 202, in
from torch._C import * # noqa: F403
ImportError: dlopen(/Users/jeffreyhomeimac/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/_C.cpython-310-darwin.so, 2): Library not loaded: @loader_path/../.dylibs/libomp.dylib
Referenced from: /Users/jeffreyhomeimac/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.dylib
Reason: no suitable image found. Did find:
/Users/jeffreyhomeimac/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/lib/../.dylibs/libomp.dylib: cannot load 'libomp.dylib' (load command 0x80000034 is unknown)
/Users/jeffreyhomeimac/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/lib/../.dylibs/libomp.dylib: cannot load 'libomp.dylib' (load command 0x80000034 is unknown)
JEFFREYs-MBP-2:stable-diffusion-webui jeffreyhomeimac$

wbf22 · 2024-05-06T14:30:29Z

I did the brew install libomp but didn't work for me. I'm working in c++ with libtorch though. But copying the installed limbomp files to the directory where my executable was solved the problem. I found them with 'brew info libomp'

kaszperro closed this as completed May 1, 2019

HiromuHota pushed a commit to HiromuHota/fonduer that referenced this issue May 2, 2019

Fix the error when importing torch (1.1.0) on osx

6de4b3a

pytorch/pytorch#20030

HiromuHota mentioned this issue May 2, 2019

Fix the error when importing torch (1.1.0) on osx HazyResearch/fonduer#251

Merged

soumith reopened this May 3, 2019

kaszperro mentioned this issue May 5, 2019

Unable to import 1.1 when installing with pip #20118

Closed

izdeby added duplicate module: build Build system issues triage review labels May 6, 2019

izdeby added high priority and removed duplicate triage review labels May 6, 2019

ezyang added module: binaries Anything related to official binaries that we release to users high priority triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module and removed high priority labels May 6, 2019

ezyang added high priority and removed high priority labels May 6, 2019

pjh5 mentioned this issue May 7, 2019

Remove brew libomp from binary mac machines #20228

Closed

facebook-github-bot closed this as completed in 9c62280 May 8, 2019

This was referenced May 15, 2019

Cost bo loop EmuKit/emukit#195

Merged

unittest fails because of pytorch bug EmuKit/emukit#197

Closed

mahaa2 mentioned this issue Jun 5, 2019

Error when trying to install pytorch / torch in apple laptop probabilisticai/probai-2019#1

Closed

twesterhout mentioned this issue Jun 18, 2019

A fix for OpenMP problems on OS X netket/netket#227

Merged

daenuprobst mentioned this issue Oct 11, 2019

importError on macOS Mojave reymond-group/tmap#1

Closed

iancze mentioned this issue Nov 19, 2019

Python binding installation difficulty on Mac OSX 10.15 flatironinstitute/finufft#55

Closed

jimfangx mentioned this issue Feb 3, 2020

Error when installing: TypeError: Class advice impossible in Python3. Use the @implementer class decorator instead. Hellisotherpeople/CX_DB8#16

Open

Borda mentioned this issue Mar 1, 2020

CI: split tests-examples Lightning-AI/pytorch-lightning#990

Merged

Brunox13 mentioned this issue Mar 6, 2020

scnym not working; Reason: image not found calico/scnym#2

Closed

ahalev mentioned this issue Apr 29, 2022

Cannot import gpboost on an M1 Mac fabsig/GPBoost#60

Closed

This was referenced Jun 3, 2022

MacOS install error: Library not loaded: @rpath/libc++.1.dylib #36941

Closed

library libshm.dylib is missing #78938

Closed

jfbu mentioned this issue Jul 30, 2022

Code does not show up in pdf since v5.1.0 sphinx-doc/sphinx#10719

Closed

pbsds mentioned this issue Apr 27, 2024

libtorch-bin: 2.0.0 -> 2.2.2 NixOS/nixpkgs#302427

Closed

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

I can't import PyTorch, libomp.dylib can't be loaded. #20030

I can't import PyTorch, libomp.dylib can't be loaded. #20030

kaszperro commented May 1, 2019 •

edited

kaszperro commented May 1, 2019

ezyang commented May 6, 2019

ezyang commented May 6, 2019

JerryShih commented May 7, 2019

ezyang commented May 7, 2019

soumith commented May 7, 2019

pjh5 commented May 7, 2019

dudongge commented May 13, 2019

gongenbo commented May 31, 2019

adrienchaton commented Jun 5, 2019

akirchhoff-modular commented Jun 5, 2019

kaszperro commented Jun 5, 2019

adrienchaton commented Jun 6, 2019

soumith commented Jun 21, 2019

sukuya commented Jul 11, 2019 •

edited

akirchhoff-modular commented Jul 11, 2019

torrmal commented Jul 13, 2019

akirchhoff-modular commented Jul 13, 2019

sukuya commented Jul 13, 2019

pjh5 commented Jul 15, 2019

GPPyeye commented May 12, 2020

leedrake5 commented Jun 20, 2020

MattioCh commented Jul 8, 2020

PearlW1 commented Dec 22, 2020

callmejliu commented Apr 3, 2023

wbf22 commented May 6, 2024

I can't import PyTorch, libomp.dylib can't be loaded. #20030

I can't import PyTorch, libomp.dylib can't be loaded. #20030

Comments

kaszperro commented May 1, 2019 • edited

🐛 Bug

To Reproduce

Expected behavior

Environment

kaszperro commented May 1, 2019

ezyang commented May 6, 2019

ezyang commented May 6, 2019

JerryShih commented May 7, 2019

ezyang commented May 7, 2019

soumith commented May 7, 2019

pjh5 commented May 7, 2019

dudongge commented May 13, 2019

gongenbo commented May 31, 2019

adrienchaton commented Jun 5, 2019

akirchhoff-modular commented Jun 5, 2019

kaszperro commented Jun 5, 2019

adrienchaton commented Jun 6, 2019

soumith commented Jun 21, 2019

sukuya commented Jul 11, 2019 • edited

akirchhoff-modular commented Jul 11, 2019

torrmal commented Jul 13, 2019

akirchhoff-modular commented Jul 13, 2019

sukuya commented Jul 13, 2019

pjh5 commented Jul 15, 2019

GPPyeye commented May 12, 2020

leedrake5 commented Jun 20, 2020

MattioCh commented Jul 8, 2020

PearlW1 commented Dec 22, 2020

callmejliu commented Apr 3, 2023

wbf22 commented May 6, 2024

kaszperro commented May 1, 2019 •

edited

sukuya commented Jul 11, 2019 •

edited