update the build of LightGBM python-package

[guolinke]

No description provided.

[nerdcha]

Hi there! I'm a little confused by this — is LightGBM not building correctly at the moment?

[guolinke]

@nerdcha
It is correct, but has the duplicate builds.

We update the installation method, it doesn't need to build the cli version first.

[nerdcha]

@guolinke Oh, I see! Thanks for correcting that, much appreciated!

[nerdcha]

Hrm, this caused a build failure, and installing via pip install lightgbm triggered a downgrade of already-installed libraries like sklearn. I've reverted this change for now.

[guolinke]

@nerdcha It is very strange. Can you provide the error log ?

[nerdcha]

Whoops: on rechecking the logs, it seems it wasn't the change itself, but the fact that LightGBM needs cmake 3.2+ now. I'm looking in to upgrading cmake from the apt-get version.

[guolinke]

okay, i see.

     cd /usr/local/src && git clone --recursive --depth 1 https://github.com/Microsoft/LightGBM && \
     cd LightGBM && mkdir build && cd build && cmake .. && make -j $(nproc) && \
     cd /usr/local/src/LightGBM/python-package && python setup.py install && \

this should be the same as

     cd /usr/local/src && git clone --recursive --depth 1 https://github.com/Microsoft/LightGBM && \
     cd /usr/local/src/LightGBM/python-package && python setup.py install && \

But the latter is faster .

[nerdcha]

Uh oh, looks like we're in DLL Hell: a python -c 'import lightgbm' throws this exception

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/conda/lib/python3.6/site-packages/lightgbm/__init__.py", line 8, in <module>
    from .basic import Booster, Dataset
  File "/opt/conda/lib/python3.6/site-packages/lightgbm/basic.py", line 31, in <module>
    _LIB = _load_lib()
  File "/opt/conda/lib/python3.6/site-packages/lightgbm/basic.py", line 26, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "/opt/conda/lib/python3.6/ctypes/__init__.py", line 426, in LoadLibrary
    return self._dlltype(name)
  File "/opt/conda/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /opt/conda/bin/../lib/libgomp.so.1: version `GOMP_4.0' not found (required by /opt/conda/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so)

@guolinke Have you come across this issue before by any chance?

[guolinke]

@nerdcha
it is installed from source ?

it seems is the version of gcc is too low, or it has multi versions of gcc.
refer to: https://stackoverflow.com/questions/39017485/openmp-linux-gomp-4-0-not-found
dmlc/xgboost#1786

[guolinke]

@nerdcha BTW, I just relax the version of cmake in LightGBM master branch. So you don't need to install the cmake now.

[nerdcha]

The weird thing is that the library file it claims to be looking in does have the symbol that it's searching for:

/# nm -D --defined-only /opt/conda/lib/libgomp.so.1 | grep GOMP_4.0
0000000000000000 A GOMP_4.0

And the default version of gcc supports gomp 4

/# gcc --version
gcc (Debian 4.9.2-10) 4.9.2

I resolved a similar issue some time in the past with a cheeky monkey patch, so I guess it's not the most stable solution. Maybe I'll try uninstalling Conda's gcc completely before adding gcc 4.9…