venv tests break virtualenv's pip when run from a -mvirtualenv virtualenv

[asottile]

Here's a reproduction, not exactly sure what's happening here:

$ tox -e py36 -r --notest
GLOB sdist-make: /home/asottile/workspace/pre-commit/setup.py
py36 create: /home/asottile/workspace/pre-commit/.tox/py36
py36 installdeps: -rrequirements-dev.txt
py36 inst: /home/asottile/workspace/pre-commit/.tox/dist/pre_commit-1.10.0.zip
py36 installed: You are using pip version 9.0.1, however version 10.0.1 is available.,You should consider upgrading via the 'pip install --upgrade pip' command.,aspy.yaml==1.1.1,atomicwrites==1.1.5,attrs==18.1.0,cached-property==1.4.2,cfgv==1.0.0,coverage==4.5.1,flake8==3.5.0,identify==1.0.18,mccabe==0.6.1,mock==2.0.0,more-itertools==4.2.0,nodeenv==1.3.0,pbr==4.0.3,pluggy==0.6.0,-e git+git@github.com:pre-commit/pre-commit@97fb49a533de9a378d20f0a41e79df118362e534#egg=pre_commit,py==1.5.3,pycodestyle==2.3.1,pyflakes==1.6.0,pytest==3.6.0,pytest-env==0.6.2,PyYAML==3.12,six==1.11.0,toml==0.9.4,virtualenv==16.0.0
___________________________________ summary ____________________________________
  py36: skipped tests
  congratulations :)

$ head -1 .tox/py36/bin/pip
#!/home/asottile/workspace/pre-commit/.tox/py36/bin/python3.6
$ .tox/py36/bin/pytest tests -k venv
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /home/asottile/workspace/pre-commit, inifile: tox.ini
plugins: env-0.6.2
collected 500 items / 492 deselected                                           

tests/repository_test.py ..                                              [ 25%]
tests/commands/install_uninstall_test.py .                               [ 37%]
tests/languages/all_test.py .....                                        [100%]

=================== 8 passed, 492 deselected in 4.12 seconds ===================
$ head -1 .tox/py36/bin/pip
#!/home/asottile/workspace/pre-commit/.tox/py36/bin/python3.6
$ tox -e py36 -- tests -k venv
GLOB sdist-make: /home/asottile/workspace/pre-commit/setup.py
py36 inst-nodeps: /home/asottile/workspace/pre-commit/.tox/dist/pre_commit-1.10.0.zip
py36 installed: You are using pip version 9.0.1, however version 10.0.1 is available.,You should consider upgrading via the 'pip install --upgrade pip' command.,aspy.yaml==1.1.1,atomicwrites==1.1.5,attrs==18.1.0,cached-property==1.4.2,cfgv==1.0.0,coverage==4.5.1,flake8==3.5.0,identify==1.0.18,mccabe==0.6.1,mock==2.0.0,more-itertools==4.2.0,nodeenv==1.3.0,pbr==4.0.3,pluggy==0.6.0,pre-commit==1.10.0,py==1.5.3,pycodestyle==2.3.1,pyflakes==1.6.0,pytest==3.6.0,pytest-env==0.6.2,PyYAML==3.12,six==1.11.0,toml==0.9.4,virtualenv==16.0.0
py36 runtests: PYTHONHASHSEED='93802395'
py36 runtests: commands[0] | coverage erase
py36 runtests: commands[1] | coverage run -m pytest tests -k venv
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /home/asottile/workspace/pre-commit, inifile: tox.ini
plugins: env-0.6.2
collected 500 items / 492 deselected                                           

tests/repository_test.py ..                                              [ 25%]
tests/commands/install_uninstall_test.py .                               [ 37%]
tests/languages/all_test.py .....                                        [100%]

=================== 8 passed, 492 deselected in 4.32 seconds ===================
py36 runtests: commands[2] | coverage report --fail-under 99
Name                                              Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------------------------
...
17 files skipped due to complete coverage.
ERROR: InvocationError: '/home/asottile/workspace/pre-commit/.tox/py36/bin/coverage report --fail-under 99'
___________________________________ summary ____________________________________
ERROR:   py36: commands failed

$ head -1 .tox/py36/bin/pip
#!/tmp/pytest-of-asottile/pytest-3/test_python_venv0/0/.pre-commit/repo5xcuq11q/py_venv-python3.6/bin/python3.6

[asottile]

Ok, I can factor tox out of the situation:

$ . .tox/py36/bin/activate
(py36) $ pytest tests -k venv
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /home/asottile/workspace/pre-commit, inifile: tox.ini
plugins: env-0.6.2
collected 500 items / 492 deselected                                           

tests/repository_test.py ..                                              [ 25%]
tests/commands/install_uninstall_test.py .                               [ 37%]
tests/languages/all_test.py .....                                        [100%]

=================== 8 passed, 492 deselected in 4.19 seconds ===================
(py36) $ head -1 .tox/py36/bin/pip
#!/tmp/pytest-of-asottile/pytest-4/test_python_venv0/0/.pre-commit/reposmsza1fi/py_venv-python3.6/bin/python3.6

[asottile]

@ojii any ideas here? (-mvenv 🕷️s)

This seems to be the minimal reproduction:

#!/usr/bin/env bash
set -euxo pipefail

tox -e py36 -r --notest
PATH=$PWD/.tox/py36/bin:$PATH
.tox/py36/bin/pytest tests/repository_test.py -k test_python_venv
head -1 .tox/py36/bin/pip

[asottile]

It seems making a -mvenv virtualenv from a -mvirtualenv virtualenv just doesn't work great:

$ virtualenv venv -ppython3.6
Running virtualenv with interpreter /usr/bin/python3.6
Using base prefix '/usr'
New python executable in /tmp/t/venv/bin/python3.6
Also creating executable in /tmp/t/venv/bin/python
Installing setuptools, pip, wheel...done.
$ . venv/bin/activate
(venv) $ python3.6 -m venv venv3
(venv) $ head -1 $(which pip)
#!/tmp/t/venv/bin/python3.6
(venv) $ ls venv
bin  include  lib  pip-selfcheck.json
(venv) $ head -n1 venv*/bin/pip
#!/tmp/t/venv/bin/python3.6
(venv) $ ls venv
venv/  venv3/ 
(venv) $ ls venv3/bin/
activate  activate.csh  activate.fish  python  python3  python3.6
(venv) $ ls -al venv3/bin/python3.6 
lrwxrwxrwx 1 asottile asottile 25 May 28 11:36 venv3/bin/python3.6 -> /tmp/t/venv/bin/python3.6

[asottile]

• pypa/virtualenv issue: py3 venv creation fails when a virtualenv is active pypa/virtualenv#1095
• bpo issue: https://bugs.python.org/issue30811