Use hatchling build backend

[fleetingbytes]

Situation

python-semver currently uses the setuptools build backend

Possible Solution/Idea

hatchling seems to be the recommended build backend now. (Tutorial uses it by default.)

Additional context

One of the benefits of switching to hatch (which uses hatchling), besides a nice venv management, would be the possibility of using hatch-semver plugin to semantically version this project.

[tomschr]

Thanks for the issue.

One reason to use setuptools was that it's already there and it was, well, mostly simple to use. Plus the "migration" (if you want to call it that way) from old setup.py to the more modern pyproject.toml was not too disruptive.

I looked briefly into the other, more modern build backends. Surely they offer a lot of other great features. However, some (most?) of them require a lot of other dependencies. For a "small" library like semver, I'm a bit hesitant to introduce them as I think a a lean build backend is "good enough" of this project. You can build it with standard Python and don't need any other packages.

For me, it seems there is not one, dominant build backend right now (probably it will never be). They all have their pros and cons. Apart from the "recommended build backend" does hatchling have other convincing benefits that setuptools doesn't have?

To be honest, I'm a bit skeptical about this change. However, I have only touched flit and poetry (which created some obscure error messages to me), so my experiences is a bit, well, not so great. 😉

I need to look into this a bit more, to get a better picture. If you have some good arguments or you could open a pull request so I can test that, that would be very helpful.

[ofek]

Hatchling maintainer here! I can do a few:

• Supports static analysis tools for editable installations by default whereas the new setuptools does not i.e. it requires enabling an option for IDEs to work.
• Uses Git-style glob patterns rather than stdlib globs to better match user expectations.
• Allows easy and explicit control of what gets shipped in the sdist target whereas with setuptools it's an objectively confusing mix of conditional wheel options and a MANIFEST.in file.

[tomschr]

Oh cool, thanks Ofek for your comments. Really appreciated your input. 👍

It sounds all good. I'm not against it, but I need to test it myself to see what benefits it brings to this project. On the "cost" side, as far as I can see, it needs some more requirements.

If someone wants to create a PR, I'm happy to look into it.

[fleetingbytes]

I have never tried flit or poetry, always had been using setuptools before. And yes, it's a hassle to change an existing project to hatch (creating new ones is very easy). I needed about a day or two to get acquainted with hatch, but: it is worth it. Things I love about hatch:

Everything in pyproject.toml

Practically all the config you need is there.

• no separate requirements.txt, requirements-dev.txt
• dynamic versioning from src/myproject/__about__.py (can be set up there)
• most modern tools have their configurations in pyproject.toml, so you would want it anyway
• pyproject.toml contains definitions of the virtual environments and scripts relevant to the project (<- greatest value, IMO)

Virtual environment definitions and management

With setuptools I used to have one manually managed virtual environment where I had all I needed to develop the project. With hatch, you can separate it in many different dedicated environments:

• virtual environment for development dependencies: I usually have wheel and towncrier there
• virtual environment for testing: defines a test matrix for several python versions, my usual dependencies pytest, pytest-cov, behave
• virtual environment for styling the code: I use black, isort there
• virtual environment for building and serving the project's documentation, e.g. with mkdocs, pdoc3 as dependencies

When things change around a lot you can tear down and rebuild these environments in no time

hatch env prune
hatch env create

and get a nice overview of them with hatch env show.

Scripts

For each virtual environment you can define scripts which you can call with hatch run <env>:<script>. Without even first switching to that environment, hatch does that for you. While developing in my default environment, I can run hatch run test:cov to test all the matrix (cov is a script in my test environment, defined as:

pytest -vx --cov-report=term-missing --cov-config=pyproject.toml --cov=src/myproject

For rebuilding the project documentation, I do hatch run docs:build and for starting the local server with it: hatch run docs:serve. I format my code with hatch run style:fmt which runs both black and isort. I even have a script to build the changelog docs:tc (with some added PowerShell and git commands)

tc = [
    "towncrier build --yes",
    "Copy-Item -path ./CHANGELOG.md -destination ./docs -force",
    "git add --update",
    "git commit --message \"updated changelog\""
]

For every environment you can define its own environment variables (e.g. as an extra config for some tools)

Building and publishing

Once you have hatch config set up, you build the project with

hatch build

(I find it useful to have the env var HATCH_BUILD_CLEAN=true)
and then you can publish in any package registry you previously defined:

hatch publish --repo testpypi
hatch publish --repo pypi
hatch publish --repo myregistry

[fleetingbytes]

If someone wants to create a PR, I'm happy to look into it.

Challenge accepted. I wanted to offer you my help with updating the test code, should you decide to do something about #339, #344, or #369, so I might as well help you here, too, as it will ease my potential upcoming contributions to this project.

[tomschr]

Challenge accepted. I wanted to offer you my help with updating the test code, [...]

That would be great! At the moment, it's a one-man show 😄 My time is as limited as everyone else's. Help is always appreciated. 👍

[tomschr]

To play devils advocate, I would like to comment on some of what you wrote before:

• "Everything in pyproject.toml"
  That is already the case. 😉 Sure we have still some bits in setup.cfg, but we could move the pytest, flake8, and pycodestyle config to pyproject.toml too. Wondering if we really need pycodestyle and it could be removed completely.
  The other question is, if we really need the setup.cfg/setup.py combination at all. If I'm not mistaken, that still here because Python 3.6 needs it. However, if we still want to support Python 3.6 depends on the outcome of the discussion Should we remove support for Python 3.6? #371
  Apart from these side questions, we have already a pyproject.toml file (although it uses the setuptools backend). Switching to hatchling does not give additional benefit in regards to the format alone.

• "no separate requirements.txt, requirements-dev.txt"
  We don't have that either. Only for the docs there is one.

• "most modern tools have their configurations in pyproject.toml, so you would want it anyway"
  That's true. However, we have a pyproject.toml alreaday. From a format standpoint, switching from setuptools backend to hatchling doesn't add more benefits.
  This is already the case for black and towncrier. I actually played a bit to move all the remaining configs from setup.cfg to pyproject.toml. Sometimes it needs an additional toml/tomli lib, but it seem to work for mypy, pytest, coverage, docformatter, and isort. Only flake8 needs another wrapper as it is not prepared to read from pyproject.toml.
  Using the new pyproject.toml is certainly beneficial, but not by just switching to hatchling.

• Virtual environment definitions and management
  Isn't that the case with tox already? Would hatchling make tox obsolete?

• Scripts
  I think, that's probably something that looks beneficial.

Hope my list doesn't sound to negative. 🙂 It shouldn't prevent you from creating a PR, quite the opposite. Convince me! I just trying to get a full picture and sort the benefits for this project and the pros and cons.

However, you haven't addressed one of my major concerns: dependencies. 😄 Maybe I over-emphasize these parts and it's not a big deal today, but I still think we should keep these things to a minimum. I see it as a con, although not a big one. That's where setuptools clearly wins. That's why I choose it. To keep it simple.

Controlling many moving targets can be a nightmare. How can we mitigate this issue? Can we freeze some versions that hatchling depends on? Would it be feasible? Or should we just rely on the latest releases? What happens when one of these dependencies break? That would make developing and/or building this project at risk, wouldn't it?
(Probably at the end of the day, it boils down to any other dependency that a project needs.)

I really see the benefits in using hatchling. As I don't have any experiences with it, I'm waiting for your pull request and curious to see this in action!

To summarize it:

• How does switching to hatchling influence tox? Do we still need it?
• Dependencies
• Look into the PR to see how it works. 🙂

[fleetingbytes]

I have no experience with tox, so I think I would just leave it untouched and have all tests and matrices managed by tox, because I don't exacty undestand what goes on there and how tox locks into your Github Actions. I will prepare a smaller test matrix with hatch, which you can try independently, and see if it can be extended to cover all you do with tox.

[tomschr]

I have no experience with tox, so I think I would just leave it untouched and have all tests and matrices managed by tox, because I don't exacty undestand what goes on there and how tox locks into your Github Actions. I will prepare a smaller test matrix with hatch, which you can try independently, and see if it can be extended to cover all you do with tox.

Tox is used in the GHA here and here. These parts needs to be replaced by the respective hatchling commands.

But yes, start with something small and we can proceed from there.

Many thanks for taking that! 👍