move to pyproject.toml

[sigma67]

Hello 👋 I just recently migrated to pyproject.toml in my own open-source project, so I thought it might be nice to help out another project with this.

This PR consolidates the project root a bit by combining 4 files into 1. Let me know if you have any concerns or issues, everything worked fine for me locally :)

[lornajane]

Thanks @sigma67 , this looks interesting. Could you say something about why this would be a better approach than the exisiting setup? I'm not super familiar with the differences between these options but I'm interested to learn why you see this change as important.

[sigma67]

Hi @lornajane , sure. pyproject.toml is the new standard for declaring dependencies and project metadata backed by PyPA.

It was introduced in PEP 518 and PEP 621. Furthermore it simplifies the project root of rst2pdf quite a bit, as you only have one file where all the project data is stored. Some tools don't support it yet (like flake8), but likely will in the future.

rst2pdf currently uses setuptools as its build system. setuptools also explains in its docs that pyproject.toml will be the new standard for project metadata.

[sigma67]

I also see that currently the tests are failing on CI, I will take a look.

[akrabat]

The Manifest.in file lists the additional files that we want to be included in the source tarball when we release using python setup.py sdist. With that file removed, how does the system know that README.rst, LICENSE.txt and CHANGES.rst are to be included in the tarball?

[sigma67]

@akrabat setuptools_scm includes all files tracked by scm (git) in the result. You can specify excludes in MANIFEST.in only (unfortunately not as part of pyproject.toml yet):

https://github.com/pypa/setuptools_scm/

I just took care of that with the new commit. The includes should now be correct, I think, feel free to verify.

[sigma67]

Also rebased onto main.

[sigma67]

Should pass all tests now.

[sigma67]

Note that MANIFEST.in is only needed if we care about what's in the sdist, i.e. tar.gz. If we just include everything we can remove that file as well.

"In my opinion the sdist is meant to be more than just a copy of the Python functionality, but is meant to be a distributable copy of the source code. I would expect someone to be able to download the sdist, extract it, and develop on the project much like they would if cloning the repo."

Note that this final “problem” applies exclusively to source distributions: builds (and wheels) are automatically limited to files living under the packages as specified by the packages parameter.

https://www.remarkablyrestrained.com/python-setuptools-manifest-in/

[sigma67]

Don't hesitate to ask if there are open questions :)

[lornajane]

At least for me personally, I don't know what the questions are. Python isn't the main tech stack for the active maintainers, and I have no way of knowing what the needs of all our dependent projects are. I am not sure how to evaluate this change for safety, and I'm unclear about the benefits beyond it being the way you'd do things if you started the project today. There doesn't seem to have been a particular bug or limitation that prompted the change, and it's not something that we've been asked by users to do.

Some of the questions that are in my mind, which are pretty generic for risky changes:

• how can we test this change?
• as well as end users, who else does this change the experience for (downstream projects, distros that package the project, who else)?
• what are the benefits?
• what are the risks and how can we responsibly evaluate them?
• do we need to follow up by reaching out for some additional expert support, or contacting some of our users to assess the impact of the change?

I want to accept the changes that will improve the project, but there's a lot here that isn't clear yet.

[sigma67]

• how can we test this change?

Ensure that all packages that are built and distributed are identical to those built without the change. Ensure that tests pass as before.

• as well as end users, who else does this change the experience for (downstream projects, distros that package the project, who else)?

it does not change the experience

• what are the benefits?

Single point of configuration for developers, not spread across many files. Future proofing the package for future Python versions and new tools that may only use pyproject.toml, as this is the new standard in the Python ecosystem. Easier to acquire new contributors as it shows that you are using latest Python standards (this one's perhaps debatable).

• what are the risks and how can we responsibly evaluate them?

Something in the build pipeline may break. For evaluation see "how can we test this change".

• do we need to follow up by reaching out for some additional expert support, or contacting some of our users to assess the impact of the change?

It probably wouldn't hurt to ask some users to test this change.

[akrabat]

This is on my list to look at, but I'm not sure when I'll get to it. Just wanted to note that it's not forgotten.

[sigma67]

No worries, it's not urgent. Perhaps you can trigger the tests in this PR to see if there any issues?