conda_env installers / providers plugin hooks

[jaimergp]

Checklist

• I added a descriptive title
• I searched open requests and couldn't find a duplicate

What is the idea?

conda env supports different types of "installers" in the environment.yml specification:

name: bla
channels:
  - conda-forge
dependencies:
  - python
  - pip
  - pip:
    - requests

Assuming dependencies has type list[str | dict[str, str], the key of the accepted mappings must be one module name under conda.env.installers. There's a base class and everything, so things are almost ready.

I'd like to see this used in practice by the plugin system. I don't think it's useful to allow random plugins to implement different tools for custom environment.yml subtypes (e.g. a custom plugin should not be able to parse a - custom: [spec] field. However, I do see the point in having different plugins to handle the pip mappings. This could be used to improve the pip support in conda env via additional plugins that allow safer interoperability, or faster solves (looking at uv).

Why is this needed?

Better flexibility for the pip section in environment.yml. Allows to iterate with plugins instead of adding more burden on conda core codebase.

What should happen?

Plugins should be able to provide a hook implementation that subclasses the installers base class, for known values. For now there's only pip but maybe in the future we see node?

Additional Context

conda env has also classes for spec files. This could also be used to implement different file formats and/or versions where new behaviour is introduced (e.g. adding pypi-dependencies instead of using a - pip: [specs] mapping.

[jezdez]

I would suggest expanding this epic to two plugin hooks:

1. env_specs - Environment specification backends, to load custom user environments.

#12952 for example, is looking to implement a fsspec based "RemoteSpec" backend.

2. env_installers - Environment installer backends, to install from standard packaging hosters, e.g. PyPI.

The existing pip variant could be grandfathered into being one of the offered variants by conda-pypi, which conda needs to continue to automatically install, to have a transition path for end users.

I don't think it's useful to allow random plugins to implement different tools for custom environment.yml subtypes (e.g. a custom plugin should not be able to parse a - custom: [spec] field.

I understand, but would think that limitation only applies to the env_installers, given that it relates to official package hosters, it's in the best interest of the conda CLI to only explicitly allow additional installers, to make sure the user experience doesn't suffer.

However, I do see the point in having different plugins to handle the pip mappings.

That's tricky, we need to provide a transition path for users, and I'd rather remove the existing broken pip support in conda in favor of conda-pypi. IMO that means there should only be one canonical conda-pypi package that can provide different backends (pip, uv, poetry, etc), all available under their respective short names, pip:, and one additional one pypi: that defaults to pip (at first).

Implementation

Both env subsystems already have backend systems in place and could trivially be configured to a) be extensible and b) provide a default backend.

I'd really like to see us move the binstar spec system out of conda, into a separate package (which we still may want to auto-install as part of miniconda/miniforge).