feat: hatch deps sync

[juftin]

Adds a new CLI command: deps sync

hatch deps sync --all

This saves me from running something like hatch env run --env docs -- python --version to initiate an environment sync. I've also found that being able to run hatch dep sync --all can be especially useful. This would be particularly helpful for my plugin, hatch-pip-compile, which would allow this command to sync all lockfiles.

This is a quick and dirty implementation, I extracted the env run implementation into a function and removed the last part where it actually runs the command. I did this in the easiest way I could come up with to demonstrate the functionality - I'd be happy to refactor if it's a feature you're interested in.

Related:

• Update installed dependencies in an environment #594
• Command to explicitly sync dependencies? #650

[ofek]

I am about to release but this will definitely go in the next, thank you!

[juftin]

I am about to release but this will definitely go in the next, thank you!

Sounds great. Super excited for the upcoming release.

Regarding hatch deps sync, I also see hatch deps add and hatch deps remove as tightly interconnected. I'd be happy to help with the implementation / brainstorm UX for those kinds of features.

`hatch deps add` example code

"""
`hatch dep add` PoC

This code leverages tomlkit to parse the pyproject.toml file which is necessary
because it preserves style and comments.
"""

import pathlib
from typing import Optional, Tuple

import httpx
import packaging.requirements
import packaging.version
import tomlkit


def get_toml_dependencies(
    toml_doc: tomlkit.TOMLDocument, environment: str
) -> Tuple[tomlkit.TOMLDocument, tomlkit.array]:
    """
    Return an environment's dependencies from a TOMLDocument.

    This function will perform some basic validation to ensure that the
    TOMLDocument is structured in a way that we expect. If the TOMLDocument
    does not contain the expected structure, a ValueError will be raised.

    The table returned with the TOMLDocument is a ref of the internal data
    structure, so any changes made to it will be reflected in the original
    document.

    Parameters
    ----------
    toml_doc: tomlkit.TOMLDocument
        The TOMLDocument to parse.
    environment: str
        The environment to parse.

    Returns
    -------
    Tuple[tomlkit.TOMLDocument, tomlkit.array]
        The TOMLDocument and the array of dependencies.
    """
    toml_data = toml_doc.copy()
    if toml_data.get("hatch"):
        hatch_toml = toml_data["hatch"]
    elif toml_data.get("tool", {}).get("hatch"):
        hatch_toml = toml_data["tool"]["hatch"]
    else:
        raise ValueError("No hatch section found in config file.")
    if not hatch_toml.get("envs", {}).get(environment):
        raise ValueError(f"No environment {environment} found in config file.")
    environment = hatch_toml["envs"][environment]
    existing_dependencies = environment.get("dependencies", tomlkit.array())
    return toml_data, existing_dependencies


def lookup_requirement(
    requirement: packaging.requirements.Requirement,
) -> packaging.requirements.Requirement:
    """
    Lookup a requirement on the PyPI API.

    When a requirement is specified without a version, the latest version is
    returned from PyPI with the `~=` specifier.
    """
    response = httpx.get(f"https://pypi.org/pypi/{requirement.name}/json")
    response.raise_for_status()
    data = response.json()
    if not requirement.specifier:
        all_releases = [packaging.version.Version(version) for version in data["releases"]]
        greatest_release = max(all_releases)
        return packaging.requirements.Requirement(f"{requirement.name}~={greatest_release}")
    else:
        if requirement.specifier not in data["releases"]:
            raise ValueError(f"Requirement {requirement} not found on PyPI.")
        return requirement


def add_dependency(
    toml_doc: tomlkit.TOMLDocument,
    requirement: packaging.requirements.Requirement,
    environment: str,
) -> Optional[tomlkit.TOMLDocument]:
    """
    Add a dependency to a TOMLDocument if necessary

    Behavior:
    - If the dependency is already present with the same specifier provided,
      no changes are made.
    - If the dependency is already present and no specifier is provided, the
      no changes are made.
    - If the dependency is already present with a different specifier provided,
      the specifier is updated.
    - If the dependency is not present, it is added with the specifier provided
      or the latest version specifier if no specifier is provided.

    Returns
    -------
    Optional[tomlkit.TOMLDocument]
        The TOMLDocument with the dependency added, or None if no changes were
        made.
    """
    toml_doc, existing_dependencies = get_toml_dependencies(
        toml_doc=toml_doc, environment=environment
    )
    requirement_list = [packaging.requirements.Requirement(dep) for dep in existing_dependencies]
    existing_requirements = {req.name: req for req in requirement_list}
    matching_requirement = existing_requirements.get(requirement.name)
    if matching_requirement and not requirement.specifier:
        return None
    elif matching_requirement and matching_requirement.specifier == requirement.specifier:
        return None
    elif matching_requirement and matching_requirement.specifier != requirement.specifier:
        existing_dependencies.remove(str(matching_requirement))
    requirement_to_add = lookup_requirement(requirement)
    existing_dependencies.append(str(requirement_to_add))
    return toml_doc


def add_requirement_to_toml(
    toml_file: pathlib.Path,
    requirement: str,
    environment: str,
) -> None:
    """
    Add a requirement to the pyproject.toml file if necessary.

    This function represents the core functionality of the `hatch dep add`
    command.
    """
    toml_data = tomlkit.parse(toml_file.read_text())
    new_requirement = packaging.requirements.Requirement(requirement)
    updated_toml = add_dependency(
        toml_doc=toml_data, requirement=new_requirement, environment=environment
    )
    if updated_toml:
        tomlkit.dump(updated_toml, toml_file.open(mode="w"))


if __name__ == "__main__":
    toml_file = pathlib.Path.home() / "git" / "hatch-pip-compile" / "pyproject.toml"
    add_requirement_to_toml(
        toml_file=toml_file,
        requirement="flake8",
        environment="lint",
    )