In virtual environment populated with `flit install -s` (editable), `python3 -m pip freeze` raises `AssertionError` on MacOS

[tombaker]

Environment

• pip version: 20.2.3
• Python version: 3.8
• OS: MacOS 10.15.7 and Ubuntu 20.04 (tested on two different machines)

I created the virtual environments with python3 -m venv ... and populated them with flit install -s (which installs a local script as "editable" analogously to python3 -m pip install --editable ..

Description

When I run `python3 -m pip freeze' within the virtual environment, the following exception is raised:

ERROR: Exception:
Traceback (most recent call last):
  File "/Users/tbaker/venvs/dcapenv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
    status = self.run(options, args)
  File "/Users/tbaker/venvs/dcapenv/lib/python3.8/site-packages/pip/_internal/commands/freeze.py", line 101, in run
    for line in freeze(**freeze_kwargs):
  File "/Users/tbaker/venvs/dcapenv/lib/python3.8/site-packages/pip/_internal/operations/freeze.py", line 67, in freeze
    req = FrozenRequirement.from_dist(dist)
  File "/Users/tbaker/venvs/dcapenv/lib/python3.8/site-packages/pip/_internal/operations/freeze.py", line 257, in from_dist
    req = direct_url_as_pep440_direct_reference(
  File "/Users/tbaker/venvs/dcapenv/lib/python3.8/site-packages/pip/_internal/utils/direct_url_helpers.py", line 49, in direct_url_as_pep440_direct_reference
    assert not direct_url.info.editable
AssertionError

@sbidoul @chrahunt I note that this assertion error was introduced a few months ago (see #7612).

Expected behavior

I expected pip to display a list of installed modules, as it does when I run pip 20.2.3 within virtual environments created prior to mid-August, e.g.:

alabaster==0.7.12
appdirs==1.4.4
astroid==2.4.2
attrs==19.3.0
Babel==2.8.0
black==19.10b0

How to Reproduce

1. Go to a Github repo with a Python script under development - or clone the small project mklists that I used for this test (and note its pyproject.toml.
2. Create a virtual environment, e.g.: python3 -m venv .venv
3. Activate the virtual environment, e.g.: source .venv/bin/activate (MacOS) or . .venv/bin/activate (Ubuntu)
4. Install flit: python3 -m pip install --upgrade flit
5. Run flit install -s to install the local code as "editable".
6. Run python3 -m pip freeze.
7. Instead of displaying a list of installed modules, an exception is raised (as described above).

Output (MacOS)

577 [master] ~/github/tombaker/mklists> python3 -m venv .venv

578 [master] ~/github/tombaker/mklists> source .venv/bin/activate

(.venv) 579 [master] ~/github/tombaker/mklists> python3 -m pip install --upgrade flit
Collecting flit
  Using cached flit-3.0.0-py3-none-any.whl (48 kB)
... etc ...
Successfully installed certifi-2020.6.20 chardet-3.0.4 docutils-0.16 flit-3.0.0 flit-core-3.0.0 idna-2.10 pytoml-0.1.21 requests-2.24.0 urllib3-1.25.10
WARNING: You are using pip version 20.2.1; however, version 20.2.3 is available.
You should consider upgrading via the '/Users/tbaker/github/tombaker/mklists/.venv/bin/python3 -m pip install --upgrade pip' command.

(.venv) 580 [master] ~/github/tombaker/mklists> python3 -m pip install --upgrade pip
...
Successfully installed pip-20.2.3

(.venv) 581 [master] ~/github/tombaker/mklists> flit install -s
Extras to install for deps 'all': {'test', 'doc', '.none'}                                                                                                           I-flit.install
Installing requirements                                                                                                                                              I-flit.install
Collecting pytest
  Using cached pytest-6.1.1-py3-none-any.whl (272 kB)
... etc ...
Successfully installed Jinja2-2.11.2 MarkupSafe-1.1.1 ... etc ...
Symlinking mklists -> /Users/tbaker/github/tombaker/mklists/.venv/lib/python3.8/site-packages/mklists                                                                I-flit.install
Writing script to /Users/tbaker/github/tombaker/mklists/.venv/bin/mklists                                                                                            I-flit.install

(.venv) 582 [master] ~/github/tombaker/mklists> python3 -m pip freeze
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/tbaker/github/tombaker/mklists/.venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
    status = self.run(options, args)
  File "/Users/tbaker/github/tombaker/mklists/.venv/lib/python3.8/site-packages/pip/_internal/commands/freeze.py", line 101, in run
    for line in freeze(**freeze_kwargs):
  File "/Users/tbaker/github/tombaker/mklists/.venv/lib/python3.8/site-packages/pip/_internal/operations/freeze.py", line 67, in freeze
    req = FrozenRequirement.from_dist(dist)
  File "/Users/tbaker/github/tombaker/mklists/.venv/lib/python3.8/site-packages/pip/_internal/operations/freeze.py", line 257, in from_dist
    req = direct_url_as_pep440_direct_reference(
  File "/Users/tbaker/github/tombaker/mklists/.venv/lib/python3.8/site-packages/pip/_internal/utils/direct_url_helpers.py", line 49, in direct_url_as_pep440_direct_reference
    assert not direct_url.info.editable
AssertionError

Note

On 13 October, I described this issue on Stackoverflow

[sbidoul]

@tombaker thanks for the report. I think that's a combination of flit generating the editable flag in direct_url.json (which is correct) and pip beeing confused because flit's notion of editable is not recognized by pip. Perhaps this assert should be replaced with a warning or nothing at all. I'll look into this.

[tombaker]

@sbidoul I'm just now seeing your response. In the meantime, I posted an issue on the Flit repo pointing out that pip freeze works in virtual environments populated using Flit 2.3.0 but not in environments populating using Flit 3.0.0.

[takluyver]

Flit 2.x didn't do anything about the direct_url info - @sbidoul added it for 3.0. So no surprise that only Flit 3 causes this. I'll trust Stéphane on what needs changing. 🙂

[pradyunsg]

@sbidoul I'd say we should this make a debug log, starting with "#" to make it a comment, in case someone puts it in a file directly.

[sbidoul]

@tombaker I made a fix in #9050. It would great if you give it a spin.

[tombaker]

@sbidoul Starting with a clean install and upgrading to Pip 20.2.4, I still get AssertionError:

535 [] /tmp> git clone https://github.com/tombaker/mklists
Cloning into 'mklists'...
536 [] /tmp> cd mklists
537 [master] /tmp/mklists> python3 -m venv .venv
538 [master] /tmp/mklists> source .venv/bin/activate
(.venv) 539 [master] /tmp/mklists> python3 -m pip install --upgrade pip
Collecting pip ...
Successfully installed pip-20.2.4
(.venv) 540 [master] /tmp/mklists> python3 -m pip install --upgrade flit
Collecting flit ...
(.venv) 541 [master] /tmp/mklists> flit install -s
Installing requirements  ...                                                                                                                                            I-
Writing script to /private/tmp/mklists/.venv/bin/mklists ...
(.venv) 542 [master] /tmp/mklists> python3 -m pip freeze
ERROR: Exception:
Traceback (most recent call last):
  File "/private/tmp/mklists/.venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
    status = self.run(options, args)
  File "/private/tmp/mklists/.venv/lib/python3.8/site-packages/pip/_internal/commands/freeze.py", line 101, in run
    for line in freeze(**freeze_kwargs):
  File "/private/tmp/mklists/.venv/lib/python3.8/site-packages/pip/_internal/operations/freeze.py", line 67, in freeze
    req = FrozenRequirement.from_dist(dist)
  File "/private/tmp/mklists/.venv/lib/python3.8/site-packages/pip/_internal/operations/freeze.py", line 257, in from_dist
    req = direct_url_as_pep440_direct_reference(
  File "/private/tmp/mklists/.venv/lib/python3.8/site-packages/pip/_internal/utils/direct_url_helpers.py", line 49, in direct_url_as_pep440_direct_reference
    assert not direct_url.info.editable
AssertionError

[sbidoul]

@tombaker sorry I this fix is not released yet. I meant to test the PR. To do so you can install it with "python -m pip install git+https://github.com/pypa/pip@refs/pull/9050/head"