ImportError: numpy.core._multiarray_umath failed to import

[yihanwu]

• spladder version: 2.1.1
• Python version: 3.5.4
• Operating System:x86_64 GNU/Linux
• Packages installed:
  cycler==0.10.0
  h5py==2.9.0
  intervaltree==3.0.0
  matplotlib==2.1.2
  numpy==1.14.5
  pandas==0.24.1
  patsy==0.5.1
  pyparsing==2.4.0
  pysam==0.15.2
  python-dateutil==2.8.0
  pytz==2019.1
  scipy==1.2.1
  six==1.12.0
  sortedcontainers==2.1.0
  spladder==2.2.1
  statsmodels==0.9.0

Description

I am attempting to install spladder on a cluster inside a virtualenv.

From running pip install spladder, everything installs properly.

What I did

However, when I attempt to run spladder on the command line, an error occurs (see below).

ImportError: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
  File "/home/yihanwu/spladded3.5/bin/spladder", line 7, in <module>
    from spladder.spladder import main
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/spladder/spladder.py", line 3, in <module>
    from .spladder_build import spladder
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/spladder/spladder_build.py", line 18, in <module>
    import h5py
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/h5py/__init__.py", line 36, in <module>
    from ._conv import register_converters as _register_converters
  File "h5py/h5t.pxd", line 14, in init h5py._conv
  File "h5py/h5t.pyx", line 1, in init h5py.h5t
SystemError: execution of module h5py.utils raised unreported exception

This appeared related to other issues such as this one so I attempted to update numpy to 1.16.2.

Another error was raised.

Traceback (most recent call last):
  File "/home/yihanwu/spladded3.5/bin/spladder", line 7, in <module>
    from spladder.spladder import main
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/spladder/spladder.py", line 5, in <module>
    from .spladder_test import spladder_test
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/spladder/spladder_test.py", line 3, in <module>
    import statsmodels.api as sm
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/statsmodels/api.py", line 7, in <module>
    from .regression.recursive_ls import RecursiveLS
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/statsmodels/regression/recursive_ls.py", line 16, in <module>
    from statsmodels.tsa.statespace.mlemodel import (
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/statsmodels/tsa/statespace/mlemodel.py", line 18, in <module>
    from .simulation_smoother import SimulationSmoother
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/statsmodels/tsa/statespace/simulation_smoother.py", line 10, in <module>
    from .kalman_smoother import KalmanSmoother
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/statsmodels/tsa/statespace/kalman_smoother.py", line 11, in <module>
    from statsmodels.tsa.statespace.representation import OptionWrapper
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/statsmodels/tsa/statespace/representation.py", line 10, in <module>
    from .tools import (
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/statsmodels/tsa/statespace/tools.py", line 207, in <module>
    set_mode(compatibility=None)
  File "/home/yihanwu/spladded3.5/lib/python3.5/site-packages/statsmodels/tsa/statespace/tools.py", line 59, in set_mode
    from . import (_representation, _kalman_filter, _kalman_smoother,
ImportError: cannot import name '_representation'

I'm not quite sure where the problem lies with the dependencies.

Any help would be appreciated!
Yihan

[akahles]

Dear Yihan,

It is difficult for me to debug specific environment issues. As a general comment - we had so far good success with using Conda for managing environments. I would recommend that you install conda and then create a conda environment for spladder like so:

conda create --name spladder_env python=3.6 pip scipy pysam matplotlib statsmodels intervaltree h5py 
source activate spladder_env
pip install spladder

I hope this helps.

Best,

Andre

[yihanwu]

Hi Andre,

Do you mind posting the versions of the packages for a working installation of spladder?

Based on the import error and the responses from the link I referenced, I need to update numpy to 1.16.2 or higher. However, spladder requires numpy<1.15.0, which causes a dependency conflict. I can install the packages independently but I would like to know which specific versions of the other packages are being used.

Thanks,
Yihan

[akahles]

Dear Yihan,

This is a minimal set of packages that works for me in a conda environment, using python 3.6:

    - pip >=19.0 
    - python =3.6
    - h5py =2.9.0
    - htslib =1.9
    - intervaltree =3.0.2
    - matplotlib =2.1.2
    - scipy =1.2.1
    - statsmodels =0.9.0
    - pysam =0.15.2

I hope this helps.

Best,
Andre

[yihanwu]

Hi Andre,

I'm still getting the same error for numpy when I install separately.

Do you mind showing me which version of numpy gets installed as a dependency for statsmodels?

Thank you for your help,
Yihan

[akahles]

Hi Yihau,

Thanks for the feedback. I did not catch one of the statements in your earlier posts. Indeed, the requirements are listed as numpy<=1.15. However, this is no longer necessary. I will drop this version requirement from the list. It should work fine with later versions. I am currently running numpy 1.16.2

Cheers,

Andre

[yihanwu]

Hi Andre,

It works!

Cheers,
Yihan