Seeking feedback on a plug-in-like system for environment discovery

[brettcannon]

I have an idea at brettcannon/python-launcher#168 for a design to make it easier to discover interpreters and environments. From a pipenv perspective, I see both the consumption and production aspect of the proposal being applicable. For consumption, I could see this helping to discover available Python interpreters for creating virtual environments. For producing, I can see pipenv telling other tools, like VS Code, about the virtual environment it has created for the user.

At this point I'm just looking back for feedback on the proposal; no expectation of implementing anything. If the feedback is positive across various tools, I hope to implement this for the Python Launcher for Unix and VS Code. I would also want to see if it could be turned into a PEP.

Thanks in advance for any feedback you choose to provide!

[matteius]

I had seen you open the topic originally but I didn't have any immediate feedback to provide--I don't use VS Code, as I prefer using JetBrains PyCharm and WebStorm for most of my work. I am also not super familiar with the Python Launcher for Unix, if that means something specific outside of the python binary? At this point we have been taking on a lot of technical debt in the project, and so plug-in system hasn't really been considered. I would need a better understanding of what the interface is like, but if I grasp the idea I think there are some challenges -- right now we map to a specific virtualenv for the project, if you can choose an alternative one that doesn't really apply to your Pipfile, what are the implications?

For pipenv, we currently interface with pyenv (and asdf though I haven't used that) and it does a good job with managing python versions. We are looking are reducing the --system flag restrictions, and there is a PR that had some traction around allowing overriding the virtualenv name (to some extent). I haven't much advice to offer on creating PEPs, but it sounds difficult--probably whatever you describe in the PEP needs to be to be agnostic to tools. PyCharm already has some pipenv connectors so I wonder what you are imagining the python-launcher's role is supposed to be?

[brettcannon]

I don't use VS Code, as I prefer using JetBrains PyCharm and WebStorm for most of my work.

My hope is this will eventually get picked up by all the major editors.

I am also not super familiar with the Python Launcher for Unix, if that means something specific outside of the python binary?

Yep, it's https://github.com/brettcannon/python-launcher ; the py command from Windows, but for Unix!

I would need a better understanding of what the interface is like, but if I grasp the idea I think there are some challenges -- right now we map to a specific virtualenv for the project, if you can choose an alternative one that doesn't really apply to your Pipfile, what are the implications?

I'm actually not proposing a plug-in system for pipenv, but that it participates in the plug-in system for the Python Launcher. So you could have an entry point like _py_launcher_pipenv which returns the JSON details for the virtual environment that pipenv created for the project. That way other tools don't have to know where pipenv puts its virtual environment and instead just tells tools directly.

I haven't much advice to offer on creating PEPs, but it sounds difficult

I'm the 4th most prolific PEP author, so that's not a concern for me. 😁

whatever you describe in the PEP needs to be to be agnostic to tools.

Correct. The entire design is being done in a tool-agnostic way, it just so happens VS Code and the Python Launcher are motivators and cover two differing use-cases.

PyCharm already has some pipenv connectors so I wonder what you are imagining the python-launcher's role is supposed to be?

The Python Launcher tries to figure what the most appropriate environment/interpreter is for the current situation. As such, it needs to know what's available. Right now that means pipenv isn't supported by the Python Launcher if the virtual environment is not placed into a .venv subdirectory. Having a tool-agnostic way for tools to announce what enviroments they know about would let pipenv participate in this (as well as telling VS Code, PyCharm, etc.), all without having to hard-code pipenv support into the tool.

[matteius]

This sounds like a great improvement -- right now the logic for finding python interpreters to use for pipenv are complicated and often can cause a variety of issues for users, especially when using the --system flag.

[matteius]

#4340