Python package partially installed in Docker with pip >= 7.0.0

[matm]

After a hard day troubleshooting my Docker deployment (django app), I finally found that latest version of pip (7.0.0 and 7.0.1) breaks the installation of Django inside Docker.

Don't know how it's related (or not), but pip install Django==1.3 works fine in a virtualenv, i.e I get all the files installed (a complete distribution):

(myenv):~/progs/argamato $ pip -V
pip 7.0.1 from /home/mathias/.virtualenvs/myenv/lib/python2.7/site-packages (python 2.7)

I can find the login.html template required on the admin page:

(myenv):~/.virtualenvs/myenv/lib/python2.7/site-packages $ find django \
  -name login.html
django/contrib/admin/templates/admin/login.html
django/contrib/auth/tests/templates/registration/login.html
django/contrib/flatpages/tests/templates/registration/login.html

while the same command run inside my Docker web container returns nothing:

root@9edf6692c07d:/src/argamato# pip -V
pip 7.0.1 from /usr/local/lib/python2.7/dist-packages (python 2.7)
root@9edf6692c07d:/src/argamato# find /usr/local/lib/python2.7/dist-packages/django \
    -name login.html
root@9edf6692c07d:/src/argamato#

Comparing size of deployed distribution is kind of fun too:

root@9edf6692c07d:/src/argamato# du -sh /usr/local/lib/python2.7/dist-packages/django
13M /usr/local/lib/python2.7/dist-packages/django

While the real uncompressed stuff takes a lot more (about 40MB).

Some investigations shows that uncompressing the cached tarball creates the django, Django-1.3.data and Django-1.3.dist-info directories:

root@9edf6692c07d:~# unzip .cache/pip/./wheels/87/59/82/4099bbbed04d9d048d7fd9d2284716df97998a2a2a1895674f/Django-1.3-py2-none-any.whl -d /tmp
root@9edf6692c07d:~# ls /tmp/
django  Django-1.3.data  Django-1.3.dist-info

My missing templates are in the /tmp/Django-1.3.data directory but pip install never deploys it in /usr/local/lib/python2.7/dist-packages/. Only django and Django-1.3.dist-info are available, thus the missing template files.

Anyway, that reminded me that my Docker installation was perfectly working a few days ago, so I switch back to pip 6.1.1 and BAM, problem solved...

Don't you think of a regression bug here?

(BTW, pip 7.0.1 inside a virtuaenv inside a Docker container is broken too).

[xavfernandez]

Yup, I've recently had a similar issue with django 1.4.3. This comes from the fact that installing these old versions of django from a .tar.gz or from a .whl wont produce the same install.

Since pip 7 installs everything through wheels (or at lease tries), you will have to explicitely tell it to stick to sdist:

pip install --no-binary django django==1.3

[dstufft]

I'm going to close this, the problem is that older versions of Django hacked at things in their setup.py files which breaks when being installed from wheels. Installing from Wheels is the future of Python packaging and, while unfortunate, it's impossible to avoid not breaking some things while moving to that future. Knowing this would cause issues we ensures that we had the ability to opt-out of wheels on a per project basis as @xavfernandez mentioned, you can simple add --no-binary project1,project2,project3 or --no-binary :all: to avoid building or using wheels.

[matm]

Okay, thanks for the tip @xavfernandez.

However @dstufft, how do you explain that pip install django==1.3 works perfectly well outside Docker (on a bare host and in a virtual env on bare host). I was mostly reporting a strange behaviour inside a Docker container.

[dstufft]

You probably don't have wheel installed?

[matm]

Indeed, but is it a documented requirement for upgrading to pip 7.0.0+? I don't have wheel installed in my (non Docker) virtual env and it works fine.

[dstufft]

It's not a requirement exactly, but if wheel is there then it'll build wheels and cache them, if wheel isn't there then it'll use the pre 7.0 method.

[macropin]

Is there a relevant Django bug for this?

[hobbsd]

I had this problem when I upgraded Django from 1.4.20 to 1.4.21 and didn't see a Django bug for it, so I filed one: https://code.djangoproject.com/ticket/25119