Issue on Pandas installation on Alpine:3.10 docker container

[jitendrs]

Code Sample, a copy-pastable example if possible

Collecting pandas
  Downloading https://files.pythonhosted.org/packages/b7/93/b544dd08092b457d88e10fc1e0989d9397fd32ca936fdfcbb2584178dd2b/pandas-0.25.3.tar.gz (12.6MB)

 ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u_c9l9_o/pandas/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u_c9l9_o/pandas/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-u_c9l9_o/pandas/pip-egg-info
         cwd: /tmp/pip-install-u_c9l9_o/pandas/
    Complete output (15 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-u_c9l9_o/pandas/setup.py", line 815, in <module>
        ext_modules=maybe_cythonize(extensions, compiler_directives=directives),
      File "/tmp/pip-install-u_c9l9_o/pandas/setup.py", line 537, in maybe_cythonize
        numpy_incl = pkg_resources.resource_filename("numpy", "core/include")
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1144, in resource_filename
        return get_provider(package_or_requirement).get_resource_filename(
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 364, in get_provider
        return _find_adapter(_provider_factories, loader)(module)
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1392, in __init__
        self.module_path = os.path.dirname(getattr(module, '__file__', ''))
      File "/usr/lib/python3.7/posixpath.py", line 156, in dirname
        p = os.fspath(p)
    TypeError: expected str, bytes or os.PathLike object, not NoneType
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Problem description

Version specification in use.
Pandas:- 0.25.3
Numpy :- 1.17.4
matplotlib:- 3.1.2
Cython:- 0.29.14
Docker Alpine:- 3.10
OS:- MacOs catalina 10.15.1

Hey Folks,
I am having problem with installation of pandas library on alpine docker container.
As error provided in code section, during docker build it failing over pandas installation.

Any help would be appreciated.

[TomAugspurger]

Do you have the necessary compilers? PyPI doesn't have a manylinux tag for alpine, so we can't upload wheels for it.

[bgehman]

Seeing a similar problem installing pandas on alpine 10, here's a reproducing one-liner:

$ docker run --rm -it python:3.7.4-alpine3.10 sh -c "apk add gcc linux-headers musl-dev && pip3 install pandas==0.25.3"

Results in: ModuleNotFoundError: No module named 'Cython'

Seems that pandas dependencies are broken.

If you install Cython first:

$ docker run --rm -it python:3.7.4-alpine3.10 sh -c "apk add gcc linux-headers musl-dev && pip3 install Cython && pip3 install pandas==0.25.3"

It errors out that Numpy isn't around... ModuleNotFoundError: No module named 'numpy'...

[TomAugspurger]

What version of pip, can can you try installing from GitHub?

On master, we have a pyproject.toml, which will get the build-dependencies (NumPy, Cython) before building pandas.

[bgehman]

Using python:3.7.4-alpine3.10

docker run --rm -it python:3.7.4-alpine3.10 sh -c "pip3 --version"
pip 19.3 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)

Using python:3.7.5-alpine.3.10 (same issues)

$ docker run --rm -it python:3.7.5-alpine3.10 sh -c "pip3 --version"                                                   
pip 19.3.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)

FWIW: this all used to work a few days ago.

[TomAugspurger]

Perhaps something changed on the NumPy side of things then? Pandas hasn't had a release recently, and the traceback indicates the exception is in getting NumPy's resources.

        numpy_incl = pkg_resources.resource_filename("numpy", "core/include")
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1144, in resource_filename
        return get_provider(package_or_requirement).get_resource_filename(
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 364, in get_provider
        return _find_adapter(_provider_factories, loader)(module)
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1392, in __init__
        self.module_path = os.path.dirname(getattr(module, '__file__', ''))
      File "/usr/lib/python3.7/posixpath.py", line 156, in dirname
        p = os.fspath(p)
    TypeError: expected str, bytes or os.PathLike object, not NoneType

[jitendrs]

@TomAugspurger Pip version 19.3.1

[jitendrs]

Closing since issue is with alpine libraries. @TomAugspurger @bgehman Thanks mate, Appreciate your help.

[bgehman]

@jitendrs Is it working for you? I'm still seeing the broken dependencies when installing pandas as mentioned at #30154 (comment) . If you have a solution, mind sharing it?

[jitendrs]

@bgehman Try to install below library manually,

RUN apk add
lapack-dev
libgcc
libquadmath
musl
libgfortran

[bgehman]

@jitendrs My one-liner, after adding your list of libraries:

$ docker run --rm -it python:3.7.4-alpine3.10 sh -c "apk add gcc linux-headers musl-dev lapack-dev libgcc libquadmath musl libgfortran && pip3 install pandas==0.25.3"

Still results in ModuleNotFoundError: No module named 'Cython'.

@TomAugspurger let me know if you want me to create a new issue. The dependency problem still remains.

[jitendrs]

Oops, sorry forgot to give you add on,

This are my docker contents.

RUN echo '***** Install dependencies'
    RUN apk upgrade --update  && \
    apk add  build-base \
        --virtual build-dependencies \
        bash \
        git \
        wget \
        emacs \
        vim \
        curl \
        make \
        lapack-dev \
        libgcc \
        libquadmath \
        musl \
        libgfortran \
        cmake \
        gcc \
        libxml2-dev  \
        libxslt-dev \
       g++ \
       libxml2 

# Install python3
RUN echo "***** install python and python dependencies"
RUN apk add python3-dev \
        python3-tkinter \
    &&  python3 -m pip install --upgrade pip \
    &&  python3 -m pip install cython numpy scipy  pandas

make sure you follow the sequence.

[TomAugspurger]

Try installing pandas master `pip3 install git+ https://github.com/pandas-dev/pandas`. If there are still issues on master, we can reopen. For 0.25 and earlier you have to have your build env sorted out before installing pandas if you're building from source.

…

On Tue, Dec 10, 2019 at 8:33 AM jitendrs ***@***.***> wrote: Oops, sorry forgot to give you add on, python3 -m pip install cython numpy scipy make sure you follow the sequence. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#30154?email_source=notifications&email_token=AAKAOIRSJSYQHJEUOQBSQD3QX6SEHA5CNFSM4JYHHSYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGPN55Q#issuecomment-564059894>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAOIUQ6XCTF7RNHREL34LQX6SEHANCNFSM4JYHHSYA> .

[bgehman]

@jitendrs I see that you have:

&&  python3 -m pip install cython numpy scipy  pandas

if the dependencies were working, you should only have to install pandas (and not explicitly install pandas dependencies: cython, numpy, etc).

[bgehman]

@TomAugspurger installing from master works:

$ docker run --rm -it python:3.7.4-alpine3.10 sh -c "apk add gcc g++ linux-headers musl-dev git && pip3 install --upgrade pip && pip3 install git+https://github.com/pandas-dev/pandas"

What I can gleam that does not work:

• installing pandas from pypi by itself. It seems that pip does figure out that numpy is a dependency, but the numpy install fails on: ModuleNotFoundError: No module named 'Cython'
• install cython first, and then try to install pandas: cython installs fine, and then the pandas install fails on: ModuleNotFoundError: No module named 'numpy'. I can not figure out why pip has different dependency resolution depending on whether cython is installed already or not -- but there is a lot of complexity in this area in panda's setup.py

What does work, install each of these independently and in order (each on their own pip install) -- if combining in to one pip-install, it'll also fail:

1. cython
2. numpy
3. pandas

As in: $ docker run --rm -it python:3.7.4-alpine3.10 sh -c "apk add gcc g++ linux-headers musl-dev && pip3 install cython && pip3 install numpy && pip3 install pandas"

What the mystery is, is this all used to work fine last week (and the prior year). I don't know what changed that broke this -- I guess it could be pip, or numpy, or pandas... It isn't the python/alpine docker image as per DockerHub that tag hasn't changed in months.

Anyhow, the problem should be easily reproducible by anybody. It takes far too long compiling numpy/pandas from source to make it worthwhile to use alpine (since there is no whl for numpy/pandas for alpine). Preferable for people to use python-slim and save themselves the massive amount of time trying compile this from source and dealing with the broken dependency resolution (docker run --rm -it python:3.7.4-slim sh -c "pip3 install pandas").