Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ImportError: No module named numpy.distutils.core (Ubuntu xgboost installation) #820

Closed
sidsvash26 opened this issue Feb 11, 2016 · 17 comments
Closed

ImportError: No module named numpy.distutils.core (Ubuntu xgboost installation) #820

sidsvash26 opened this issue Feb 11, 2016 · 17 comments

Comments

@sidsvash26
Copy link

I am a new user to Ubuntu and recently downloaded Ubuntu 14.04 desktop version alongside Windows 10.

My PC configs are: 4 GB RAM, 64 bit

I installed Anaconda using the following command from the terminal (after downloading from website):
bash Anaconda3-2.5.0-Linux-x86_64.sh

Installation was successful.

Now in my ubuntu terminal, I followed steps as instructed on http://xgboost.readthedocs.org/en/latest/build.html#building-on-ubuntu-debian

  1. Installed a recent GNU C++ compiler -->successful
  2. git clone --recursive https://github.com/dmlc/xgboost cd xgboost; make -j4
    Building was also successful and I could build 'libxgboost.so'
  3. sudo apt-get install python-setuptools (Successful)
  4. cd python-package
  5. sudo python setup.py install

The 5. command returns an error after a number of lines. The error is as follows:

'ImportError: No module named numpy.distutils.core'

Can anyone suggest how to get rid of this error so that I can install xgboost ?

The full log after step 5. is as follows:

sidvash@sidvash-HP-Pavilion-g6-Notebook-PC:~/xgboost/python-package$ sudo python setup.py install
Install libxgboost from: ['/home/sidvash/xgboost/python-package/xgboost/../../lib/libxgboost.so']
running install
running bdist_egg
running egg_info
writing requirements to xgboost.egg-info/requires.txt
writing xgboost.egg-info/PKG-INFO
writing top-level names to xgboost.egg-info/top_level.txt
writing dependency_links to xgboost.egg-info/dependency_links.txt
reading manifest file 'xgboost.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '' under directory 'xgboost/include'
warning: no files found matching '' under directory 'xgboost/src'
warning: no previously-included files matching 'xgboost/build/' found anywhere in distribution
warning: no previously-included files matching 'xgboost/.o' found anywhere in distribution
warning: no previously-included files matching '.pyo' found anywhere in distribution
warning: no previously-included files matching '_.pyc' found anywhere in distribution
writing manifest file 'xgboost.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/libpath.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/VERSION -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/build-python.sh -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/init.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/training.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/compat.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/core.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/sklearn.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/plotting.py -> build/bdist.linux-x86_64/egg/xgboost
byte-compiling build/bdist.linux-x86_64/egg/xgboost/libpath.py to libpath.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/training.py to training.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/compat.py to compat.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/core.py to core.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/sklearn.py to sklearn.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/plotting.py to plotting.pyc
installing package data to build/bdist.linux-x86_64/egg
running install_data
copying /home/sidvash/xgboost/python-package/xgboost/../../lib/libxgboost.so -> build/bdist.linux-x86_64/egg/xgboost
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating 'dist/xgboost-0.4-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing xgboost-0.4-py2.7.egg
removing '/usr/local/lib/python2.7/dist-packages/xgboost-0.4-py2.7.egg' (and everything under it)
creating /usr/local/lib/python2.7/dist-packages/xgboost-0.4-py2.7.egg
Extracting xgboost-0.4-py2.7.egg to /usr/local/lib/python2.7/dist-packages
xgboost 0.4 is already the active version in easy-install.pth

Installed /usr/local/lib/python2.7/dist-packages/xgboost-0.4-py2.7.egg
Processing dependencies for xgboost==0.4
Searching for scipy
Reading https://pypi.python.org/simple/scipy/
Best match: scipy 0.17.0
Downloading https://pypi.python.org/packages/source/s/scipy/scipy-0.17.0.zip#md5=28a4fe29e980804db162524f10873211
Processing scipy-0.17.0.zip
Writing /tmp/easy_install-2YPTnH/scipy-0.17.0/setup.cfg
Running scipy-0.17.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-2YPTnH/scipy-0.17.0/egg-dist-tmp-z4NE5c
Traceback (most recent call last):
File "setup.py", line 39, in
url='https://github.com/dmlc/xgboost')
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 73, in run
self.do_egg_install()
File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 96, in do_egg_install
cmd.run()
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 381, in run
self.easy_install(spec, not self.no_deps)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 597, in easy_install
return self.install_item(None, spec, tmpdir, deps, True)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 648, in install_item
self.process_distribution(spec, dist, deps)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 694, in process_distribution
[requirement], self.local_index, self.easy_install
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 620, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 858, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 870, in obtain
return installer(requirement)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 616, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 646, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 834, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1040, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1025, in run_setup
run_setup(setup_script, args)
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 50, in run_setup
lambda: execfile(
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 100, in run
return func()
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 52, in
{'file':setup_script, 'name':'main'}
File "setup.py", line 265, in

File "setup.py", line 253, in setup_package

ImportError: No module named numpy.distutils`_
@brickerino
Copy link

Are you sure you are using right python?

Try

which python

or

readlink -f `which python`

to determine whether you are using Anaconda or not.

@sidsvash26
Copy link
Author

Hi,

Thanks for commenting.

which python returns the following:
/home/sidvash/anaconda3/bin/python

@sidsvash26
Copy link
Author

@brickerino : I have just updated the full log after step 5 in the description. Please see if that helps.

@phunterlau
Copy link
Contributor

The message shows you are not having numpy, please check if your PYTHON_PATH is pointing to the anaconda package path. Also, be sure conda install numpy if not yet

@sidsvash26
Copy link
Author

@phunterlau :
I checked the python_path using the follwing command:
python -c "import sys; print(sys.path)"

It returns:
['', '/home/sidvash/anaconda3/lib/python35.zip', '/home/sidvash/anaconda3/lib/python3.5', '/home/sidvash/anaconda3/lib/python3.5/plat-linux', '/home/sidvash/anaconda3/lib/python3.5/lib-dynload', '/home/sidvash/anaconda3/lib/python3.5/site-packages', '/home/sidvash/anaconda3/lib/python3.5/site-packages/Sphinx-1.3.5-py3.5.egg', '/home/sidvash/anaconda3/lib/python3.5/site-packages/cryptography-1.0.2-py3.5-linux-x86_64.egg', '/home/sidvash/anaconda3/lib/python3.5/site-packages/setuptools-19.6.2-py3.5.egg']

This looks fine to me. However, I see in my first post that the path "/usr/lib/python2.7/dist-packages" appears in a number of lines in the error and I am not sure if this should also be there in the python path or not. Could you suggest anything?

I also ran "conda install numpy" which returned the following:

Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ....
Solving package specifications: ......................
Package plan for installation in environment /home/sidvash/anaconda3:

The following packages will be downloaded:

package build
wheel-0.29.0 py35_0 82 KB

The following packages will be UPDATED:

wheel: 0.26.0-py35_1 --> 0.29.0-py35_0

Proceed ([y]/n)? y

Fetching packages ...
wheel-0.29.0-p 100% |################################| Time: 0:00:01 50.38 kB/s
Extracting packages ...
[ COMPLETE ]|###################################################| 100%
Unlinking packages ...
[ COMPLETE ]|###################################################| 100%
Linking packages ...
[ COMPLETE ]|###################################################| 100

Tried sudo python setup.py install after installing this but no luck, still the same error

@phunterlau
Copy link
Contributor

well, you have python from the system and anaconda, and python 2 3 both, and you have system's setuptool, so some env has messed up. i can suggest creating a virtualenv and install numpy scikit-learn and xgboost

@sidsvash26
Copy link
Author

@phunterlau : Thanks for your comment. I really appreciate your help.

I am a novice on Ubuntu and don't want to mess up things further.

Could you please tell me a step by step process to do this?

I know that I can create a virtualenv using the following: (installing virtualenv is not recommended with conda)
conda create -n <env_name> python=3.5 anaconda

And then how should I install numpy scikit-learn and xgboost on this env ?

I can activate the env by:
source activate <env_name>

Should I just then proceed with further installations as follows:
pip install numpy
pip install scikit-learn

And for xgboost I should repeat all these steps?
git clone --recursive https://github.com/dmlc/xgboost cd xgboost; make -j4
cd python-package; sudo python setup.py install

@sidsvash26
Copy link
Author

Okay, so I did the following but still getting the exact same error:
conda create -n snakes python=3.5 numpy pandas scikit-learn
sidvash:~$ source activate snakes
discarding /home/sidvash/anaconda3/bin from PATH
prepending /home/sidvash/anaconda3/envs/snakes/bin to PATH

cd xgboost/python-package
sudo python setup.py install

For some reason, the installation still keeps referring to "/usr/lib/python2.7/dist-packages"

@sidsvash26
Copy link
Author

I got it sorted out by following the below steps:

  1. Remove current dir named "xgboost" which I earlier downloaded from github.
  2. pip install xgboost

This worked like a charm. I didn't try this earlier as it was mentioned on the github site that you should install from github for latest version.

I posted this question on stack overflow and someone suggested this. Here's the link:
http://stackoverflow.com/questions/35332300/importerror-no-module-named-numpy-distutils-core-ubuntu-xgboost-installation/35493714#35493714

Anyway, thanks a lot for your help guys: @brickerino @phunterlau

@phunterlau
Copy link
Contributor

Glad that pip install xgboost helped :-)

@knguyen0105
Copy link

If 'pip install xgboost' doesn't work for you, this issue can also be fixed by doing

sudo -s
python setup.py install

The problem is 'sudo python' will use system's Python instead of anacoda's python.

More detail: http://stackoverflow.com/questions/30626160/anaconda-python-not-available-from-sudo

@ashish1302
Copy link

@knguyen0105 Good worked for me....thanks man....:)

@ashish1302
Copy link

ashish1302 commented Nov 25, 2016 •
edited

@knguyen0105 giving error
"/home/ashish/anaconda2/bin/../lib/libgomp.so.1: version `GOMP_4.0' not found (required by /home/ashish/anaconda2/lib/python2.7/site-packages/xgboost/./lib/libxgboost.so)"

can anybody help?

@brando90
Copy link

what is PYTHON_PATH suppose to return?

@TiagoMHCSantana
Copy link

@knguyen0105 It worked just fine for me! Thanks!

@zebrabug
Copy link

zebrabug commented Feb 14, 2017 •
edited

Thanks knguyen0105, Summary for Ubuntu, anaconda 2:

  1. http://xgboost.readthedocs.io/en/latest/build.html
  2. sudo -s
    python setup.py install
    instead of
    cd python-package; sudo python setup.py install

@Skyfacon
Copy link

＠knguyen0105， really thanks a lot

@lock lock bot locked as resolved and limited conversation to collaborators Oct 25, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@brickerino @phunterlau @brando90 @terrytangyuan @knguyen0105 @sidsvash26 @zebrabug @ashish1302 @Skyfacon @TiagoMHCSantana