Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImportError: numpy.core._multiarray_umath failed to import #11871

Closed
sadatnfs opened this issue Sep 3, 2018 · 36 comments
Closed

ImportError: numpy.core._multiarray_umath failed to import #11871

sadatnfs opened this issue Sep 3, 2018 · 36 comments

Comments

@sadatnfs
Copy link

sadatnfs commented Sep 3, 2018

Hello,

There's a bunch of packages which I'm installing that requires NumPy (for example TensorFlow), and I can't seem to be able to install and import them because of an ImportError on the numpy side.

This is happening with both pip install and building from source.

Reproducing code example:

import numpy as np
import tensorflow as tf

Error message:

ImportError: numpy.core._multiarray_umath failed to import

Numpy/Python version information:

'1.16.0.dev0+cdbf35b'
@sadatnfs
Copy link
Author

sadatnfs commented Sep 3, 2018

I feel like this could be some system issue as well, because even an older numpy is giving me this same error

@mattip
Copy link
Member

mattip commented Sep 3, 2018

We need more information to help you. Does import numpy as np succeed and then import tensorflow as tf fail? If it is the first, then there should be more information about the exception. If it is the second, where did you get tensorflow? Which python are you using?

@sadatnfs
Copy link
Author

sadatnfs commented Sep 3, 2018

Hi @mattip:

It's the latter (as in: I can import numpy all good, but then TF is the one that's failing to import).

Here's the full trace (I am using the latest Miniconda shell file):

Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import numpy as np

In [2]: import tensorflow as tf
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
ImportError: numpy.core._multiarray_umath failed to import
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
ImportError: numpy.core._multiarray_umath failed to import
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
ImportError: numpy.core._multiarray_umath failed to import
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
ImportError: numpy.core._multiarray_umath failed to import

In [3]: np.__version__
Out[3]: '1.15.1'

@sadatnfs
Copy link
Author

sadatnfs commented Sep 3, 2018

I've always been building Numpy and Scipy from source since I wanted to link to my local MKL, but when I started getting the ImportError, I started fiddling around with pip installing numpy but this error is still there

@charris
Copy link
Member

charris commented Sep 3, 2018

In [3]: np.__version__
Out[3]: '1.15.1'

That's strange, the _multiarray_umath module is new in master. That said, I suspect that the recent code rearrangement is at the root of your problems.

@charris
Copy link
Member

charris commented Sep 3, 2018

If you are using development numpy, might want to recompile tensorflow if that is possible.

@charris
Copy link
Member

charris commented Sep 3, 2018

And maybe delete all installed non-system numpy versions and reinstall.

@sadatnfs
Copy link
Author

sadatnfs commented Sep 3, 2018

Thanks @charris ! It almost certainly looks like there was some bad cache causing this import error; I just built a clean Docker and just tested numpy (both pip and build) and Tensorflow (just pip for now) imports, and both versions of numpy checked out all good when importing TF!

I'll continue to install other packages and report back on whether it really was a bad coincidence, or whether there's some other packages doing something naughty along the way!

@sadatnfs
Copy link
Author

sadatnfs commented Sep 3, 2018

Yep things are all good! Thanks for the fast replies, closing now (still not sure why things went wrong, but at least it works now!).

@sadatnfs sadatnfs closed this as completed Sep 3, 2018
@charris
Copy link
Member

charris commented Sep 4, 2018

Link to #10915 code reorganization PR.

@amehrdad88
Copy link

Hi, facing the same issue with Django (warning with the exact same error), and any re-install, clear cache (pip) not working. Any idea why this is happening?
thanks!

@rgommers
Copy link
Member

rgommers commented Jan 2, 2019

@amehrdad88 this is unrelated to Django. What matters is how you installed Python, and how you installed NumPy and the other packages you use that depend on NumPy. Check you are using what you think you are using (e.g. everything from the same virtualenv / conda env) by looking at tracebacks or inspecting numpy.__file__.

@amehrdad88
Copy link

@amehrdad88 this is unrelated to Django. What matters is how you installed Python, and how you installed NumPy and the other packages you use that depend on NumPy. Check you are using what you think you are using (e.g. everything from the same virtualenv / conda env) by looking at tracebacks or inspecting numpy.__file__.

Ok thanks I will check though I just used the Python installer from the official website, then virtualenv and everything else from pip so shouldn't pose any major problem! I'll try reinstalling everything.

@amehrdad88
Copy link

@rgommers seems to be a problem with other packages but not sure which. I re-installed everything and had the same problem, so I just created a test venv with only numpy and tensorflow; importing tensorflow works. When I then install the other packages (with pip) from a requirements.txt file the error appears again when I try to import tensorflow.

Could it be that one of the packages being loaded is has an error with regards to numpy in it? Would this package be loaded even if I only import tensorflow?

Note: I also had a warning "FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated" which went away by updating h5py

@mattip
Copy link
Member

mattip commented Jan 2, 2019

@amehrdad88 this issue is closed. Please document exactly what commands you are executing, and try to import tensorflow after each step subsequent to installing tensorflow. If you still think the issue is with NumPy and not another package, open a new issue and attach the documentation together with the output.

@rgommers
Copy link
Member

rgommers commented Jan 2, 2019

Could it be that one of the packages being loaded is has an error with regards to numpy in it? Would this package be loaded even if I only import tensorflow?

no, that is quite unlikely. what is happening is probably pip reinstalling numpy somehow because it's pulled in via requirements.txt. Or you have a package on your PYTHONPATH somewhere. Or . If you go through and import packages one by one you should be able to find which one is problematic, and if it's pulling in something from outside your clean virtualenv.

@sklarsa
Copy link

sklarsa commented Jan 9, 2019

I am starting to get this behavior when building my application on CircleCI. In the shell:

Python 2.7.14 (default, Sep 18 2017, 00:00:00)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> import pandas as pd
ImportError: No module named _multiarray_umath
ImportError: No module named _multiarray_umath
ImportError: No module named _multiarray_umath
ImportError: No module named _multiarray_umath
>>> np.__file__
'/usr/local/lib/python2.7/dist-packages/numpy/__init__.pyc'
>>> pd.__file__
'/usr/local/lib/python2.7/dist-packages/pandas/__init__.pyc'

This build was working fine 4 days ago, but now my build is failing with no changes to requirements.txt at all. Did something recently change? I'm running numpy 1.14.3 and pandas 0.20.3

@sklarsa
Copy link

sklarsa commented Jan 9, 2019

Just as a followup, pip installing numpy first (separately), and then pandas did the trick for me...

@bibinmjose
Copy link

I am starting to get this behavior when building my application on CircleCI. In the shell:

Python 2.7.14 (default, Sep 18 2017, 00:00:00)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> import pandas as pd
ImportError: No module named _multiarray_umath
ImportError: No module named _multiarray_umath
ImportError: No module named _multiarray_umath
ImportError: No module named _multiarray_umath
>>> np.__file__
'/usr/local/lib/python2.7/dist-packages/numpy/__init__.pyc'
>>> pd.__file__
'/usr/local/lib/python2.7/dist-packages/pandas/__init__.pyc'

This build was working fine 4 days ago, but now my build is failing with no changes to requirements.txt at all. Did something recently change? I'm running numpy 1.14.3 and pandas 0.20.3

i have same problem, everything was working fine 3 days back. Now suddenly there is this import error when i try to load pickled files

@mattip
Copy link
Member

mattip commented Jan 21, 2019

The c-extension module _multiarray_umath is new to numpy 1.16.0, and may indicate a mix of versions, or version expectations, somewhere. Note this issue is closed, if you believe there is a problem with numpy please open a new issue detailing exactly what you are doing.

@loftusa
Copy link

loftusa commented Jan 25, 2019

I am getting a similar issue when trying to load pickled files:

with open(fl[0], 'rb') as f:
    pickle.load(f)
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-24-571299641ef6> in <module>
      1 with open(fl[0], 'rb') as f:
----> 2     pickle.load(f)

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

@rileypeterson
Copy link

rileypeterson commented Jan 29, 2019

Maybe this should be reopened. Not sure if this is relevant.

Also getting this when trying to use pd.read_pickle:

Full Traceback
pd.read_pickle('/path/to/a/dataframe/foo.df')
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in try_read(path, encoding)
    165                 # We want to silencce any warnings about, e.g. moved modules.
--> 166                 return read_wrapper(lambda f: pkl.load(f))
    167         except Exception:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in read_wrapper(func)
    148         try:
--> 149             return func(f)
    150         finally:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in <lambda>(f)
    165                 # We want to silencce any warnings about, e.g. moved modules.
--> 166                 return read_wrapper(lambda f: pkl.load(f))
    167         except Exception:

ImportError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in try_read(path, encoding)
    170                 return read_wrapper(
--> 171                     lambda f: pc.load(f, encoding=encoding, compat=False))
    172             # compat pickle

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in read_wrapper(func)
    148         try:
--> 149             return func(f)
    150         finally:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in <lambda>(f)
    170                 return read_wrapper(
--> 171                     lambda f: pc.load(f, encoding=encoding, compat=False))
    172             # compat pickle

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/compat/pickle_compat.py in load(fh, encoding, compat, is_verbose)
    211 
--> 212         return up.load()
    213     except:

~/anaconda3/envs/py35/lib/python3.5/pickle.py in load(self)
   1042                 assert isinstance(key, bytes_types)
-> 1043                 dispatch[key[0]](self)
   1044         except _Stop as stopinst:

~/anaconda3/envs/py35/lib/python3.5/pickle.py in load_stack_global(self)
   1350             raise UnpicklingError("STACK_GLOBAL requires str")
-> 1351         self.append(self.find_class(module, name))
   1352     dispatch[STACK_GLOBAL[0]] = load_stack_global

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/compat/pickle_compat.py in find_class(self, module, name)
    134             module, name = _class_locations_map.get(key, key)
--> 135             return super(Unpickler, self).find_class(module, name)
    136 

~/anaconda3/envs/py35/lib/python3.5/pickle.py in find_class(self, module, name)
   1391                 module = _compat_pickle.IMPORT_MAPPING[module]
-> 1392         __import__(module, level=0)
   1393         if self.proto >= 4:

ImportError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in read_pickle(path, compression)
    176     try:
--> 177         return try_read(path)
    178     except:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in try_read(path, encoding)
    174                 return read_wrapper(
--> 175                     lambda f: pc.load(f, encoding=encoding, compat=True))
    176     try:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in read_wrapper(func)
    148         try:
--> 149             return func(f)
    150         finally:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in <lambda>(f)
    174                 return read_wrapper(
--> 175                     lambda f: pc.load(f, encoding=encoding, compat=True))
    176     try:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/compat/pickle_compat.py in load(fh, encoding, compat, is_verbose)
    211 
--> 212         return up.load()
    213     except:

~/anaconda3/envs/py35/lib/python3.5/pickle.py in load(self)
   1042                 assert isinstance(key, bytes_types)
-> 1043                 dispatch[key[0]](self)
   1044         except _Stop as stopinst:

~/anaconda3/envs/py35/lib/python3.5/pickle.py in load_stack_global(self)
   1350             raise UnpicklingError("STACK_GLOBAL requires str")
-> 1351         self.append(self.find_class(module, name))
   1352     dispatch[STACK_GLOBAL[0]] = load_stack_global

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/compat/pickle_compat.py in find_class(self, module, name)
    134             module, name = _class_locations_map.get(key, key)
--> 135             return super(Unpickler, self).find_class(module, name)
    136 

~/anaconda3/envs/py35/lib/python3.5/pickle.py in find_class(self, module, name)
   1391                 module = _compat_pickle.IMPORT_MAPPING[module]
-> 1392         __import__(module, level=0)
   1393         if self.proto >= 4:

ImportError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in try_read(path, encoding)
    165                 # We want to silencce any warnings about, e.g. moved modules.
--> 166                 return read_wrapper(lambda f: pkl.load(f))
    167         except Exception:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in read_wrapper(func)
    148         try:
--> 149             return func(f)
    150         finally:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in <lambda>(f)
    165                 # We want to silencce any warnings about, e.g. moved modules.
--> 166                 return read_wrapper(lambda f: pkl.load(f))
    167         except Exception:

ImportError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in try_read(path, encoding)
    170                 return read_wrapper(
--> 171                     lambda f: pc.load(f, encoding=encoding, compat=False))
    172             # compat pickle

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in read_wrapper(func)
    148         try:
--> 149             return func(f)
    150         finally:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in <lambda>(f)
    170                 return read_wrapper(
--> 171                     lambda f: pc.load(f, encoding=encoding, compat=False))
    172             # compat pickle

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/compat/pickle_compat.py in load(fh, encoding, compat, is_verbose)
    211 
--> 212         return up.load()
    213     except:

~/anaconda3/envs/py35/lib/python3.5/pickle.py in load(self)
   1042                 assert isinstance(key, bytes_types)
-> 1043                 dispatch[key[0]](self)
   1044         except _Stop as stopinst:

~/anaconda3/envs/py35/lib/python3.5/pickle.py in load_stack_global(self)
   1350             raise UnpicklingError("STACK_GLOBAL requires str")
-> 1351         self.append(self.find_class(module, name))
   1352     dispatch[STACK_GLOBAL[0]] = load_stack_global

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/compat/pickle_compat.py in find_class(self, module, name)
    134             module, name = _class_locations_map.get(key, key)
--> 135             return super(Unpickler, self).find_class(module, name)
    136 

~/anaconda3/envs/py35/lib/python3.5/pickle.py in find_class(self, module, name)
   1391                 module = _compat_pickle.IMPORT_MAPPING[module]
-> 1392         __import__(module, level=0)
   1393         if self.proto >= 4:

ImportError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
<ipython-input-44-2ad6813d006d> in <module>()
----> 1 pd.read_pickle('/Users/riley/main/build_0003/reps/lobsang/konami-api/src/app/v0/foo.df')

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in read_pickle(path, compression)
    178     except:
    179         if PY3:
--> 180             return try_read(path, encoding='latin1')
    181         raise
    182 

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in try_read(path, encoding)
    173             except:
    174                 return read_wrapper(
--> 175                     lambda f: pc.load(f, encoding=encoding, compat=True))
    176     try:
    177         return try_read(path)

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in read_wrapper(func)
    147                             is_text=False)
    148         try:
--> 149             return func(f)
    150         finally:
    151             for _f in fh:

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/io/pickle.py in <lambda>(f)
    173             except:
    174                 return read_wrapper(
--> 175                     lambda f: pc.load(f, encoding=encoding, compat=True))
    176     try:
    177         return try_read(path)

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/compat/pickle_compat.py in load(fh, encoding, compat, is_verbose)
    210         up.is_verbose = is_verbose
    211 
--> 212         return up.load()
    213     except:
    214         raise

~/anaconda3/envs/py35/lib/python3.5/pickle.py in load(self)
   1041                     raise EOFError
   1042                 assert isinstance(key, bytes_types)
-> 1043                 dispatch[key[0]](self)
   1044         except _Stop as stopinst:
   1045             return stopinst.value

~/anaconda3/envs/py35/lib/python3.5/pickle.py in load_stack_global(self)
   1349         if type(name) is not str or type(module) is not str:
   1350             raise UnpicklingError("STACK_GLOBAL requires str")
-> 1351         self.append(self.find_class(module, name))
   1352     dispatch[STACK_GLOBAL[0]] = load_stack_global
   1353 

~/anaconda3/envs/py35/lib/python3.5/site-packages/pandas/compat/pickle_compat.py in find_class(self, module, name)
    133             key = (module, name)
    134             module, name = _class_locations_map.get(key, key)
--> 135             return super(Unpickler, self).find_class(module, name)
    136 
    137 else:

~/anaconda3/envs/py35/lib/python3.5/pickle.py in find_class(self, module, name)
   1390             elif module in _compat_pickle.IMPORT_MAPPING:
   1391                 module = _compat_pickle.IMPORT_MAPPING[module]
-> 1392         __import__(module, level=0)
   1393         if self.proto >= 4:
   1394             return _getattribute(sys.modules[module], name)[0]

ImportError: No module named 'numpy.core._multiarray_umath'

@mattip
Copy link
Member

mattip commented Jan 29, 2019

@rileypeterson this may be a duplicate of issue #12837. In any case, the pickling problems with 1.16.0 are fixed by #12842 (backported to 1.16.1 in #12869). It will be solved with the release of 1.16.1.

@ARIFPATHAN111503050
Copy link

pip install --upgrade numpy

this removes the previous version and install the newer version .. this solved the issued for me

@Masum58
Copy link

Masum58 commented Feb 21, 2019

I get the same problem, numpy.core._multiarray_umath failed to import

rajadain added a commit to WikiWatershed/model-my-watershed that referenced this issue Feb 22, 2019
A recent release of NumPy introduced a new module _multiarray_umath.
Unfortunately, there are some tools that depend on NumPy and pull
in the latest version as a build dependency. This latest version is
then replaced with the version specified in our requirements.txt,
thus the module is not available at run-time causing the segfaults.

By installing NumPy separately before requirements.txt is gathered,
we ensure that whatever was using the most recent version of NumPy
in its installation now uses the pre-installed one.

We'll have to ensure that the version of NumPy in requirements.txt
matches the one in Ansible.

For details see:

numpy/numpy#11871
https://stackoverflow.com/q/54153886
rajadain added a commit to WikiWatershed/model-my-watershed that referenced this issue Feb 22, 2019
A recent release of NumPy introduced a new module _multiarray_umath.
Unfortunately, there are some tools that depend on NumPy and pull
in the latest version as a build dependency. This latest version is
then replaced with the version specified in our requirements.txt,
thus the module is not available at run-time causing the segfaults.

By installing NumPy separately before requirements.txt is gathered,
we ensure that whatever was using the most recent version of NumPy
in its installation now uses the pre-installed one.

We'll have to ensure that the version of NumPy in requirements.txt
matches the one in Ansible.

For details see:

numpy/numpy#11871
https://stackoverflow.com/q/54153886
@Gpwner
Copy link

Gpwner commented Mar 12, 2019

I finally figure it out,It is because the numpy version is too low,upgrade numpy to 1.16.2,and everything is fine

@qynkws
Copy link

qynkws commented Mar 13, 2019

numpy的卸载再更新升级

@liuzc188
Copy link

pip install -U numpy升级numpy 就可以了

@Emesgee
Copy link

Emesgee commented Mar 19, 2019

This did the trick :)
pip install --upgrade numpy

@Andre3582
Copy link

in anaconda prompt:
pip uninstall numpy

then:
pip install numpy

worked for me!

@n0Armin
Copy link

n0Armin commented Mar 26, 2019

Hello,
There's a bunch of packages which I'm installing that requires NumPy (for example TensorFlow), and I can't seem to be able to install and import them because of an ImportError on the numpy side.
This is happening with both pip install and building from source.
Reproducing code example:
import numpy as np
import tensorflow as tf
Error message:
ImportError: numpy.core._multiarray_umath failed to import

Numpy/Python version information:
'1.16.0.dev0+cdbf35b'

I have faced the same issue.
I have solved the problem by uninstalling Theano, tensorflow, and keras; then installed them in this order:

  1. theano
  2. tensorflow
  3. keras

@datianshi21
Copy link

numpy的卸载再更新升级

就服你!别人英文提问,你中文回答!

@mattip
Copy link
Member

mattip commented Apr 23, 2019

@datianshi21 our official language is English. Google translate says your comment is "Numpy uninstall and update
Just serve you! Other people ask questions in English, you answer Chinese!"
Are you a bot?

@arun-gupta
Copy link

pip install --upgrade numpy --user gave me the error:

mxnet-mkl 1.4.0 has requirement numpy<1.15.0,>=1.8.2, but you'll have numpy 1.16.3 which is incompatible.

Everything worked after uninstalling mxnet-mkl.

@emmanueltsukerman
Copy link

It started working for me when I installed pandas. Not idea why.

@bibinmjose
Copy link

#11871 (comment)

Pandas install numpy as dependency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests