Add pyproject.toml and unify PyPI CI builds #1407

rockstorm101 · 2024-01-26T17:36:35Z

Two major changes on this PR:

Modernize build procedure to use a 'pyproject.toml' file. The idea being to keep all static project metadata within 'pyproject.toml' while all build configuration and dynamic metadata remains at 'setup.py'. The reason for this is to comply with PEP 517 and PEP 518.
Integrate all current CI actions building wheels and sdist for PyPI into a single file using cibuildwheel. The use of the current action 'RalfG/python-wheels-manylinux-build' is deprecated in favor of PyPA/cibuildwheel. Hopefully this merge will ease maintenance of the PyPI builds plus it allows us to easily produce wheels to support Apple Silicon and ARM architectures.

I labeled this PR as a Work In Progress as it will need testing particularly on Windows and macOS before it can be merged with confidence. I did not detect any issues on Linux so far, but I did not test wheels on architectures other than amd64. If it is too much of a change it can wait to be integrated on the next release cycle.

As always, this is open for discussion. Any and every feedback is appreciated.

Fixes kliment#1319

The philosophy being to keep all static project metadata within 'pyproject.toml' while all build configuration and dynamic metadata remains at 'setup.py'. The 'long_description' is no longer needed because we link to the README on 'pyproject.toml'. A 'license' field is not required because it is already set on 'classifiers'.

This way no code is run when 'setup.py' is loaded until the setup function is called.

The in-place build of the 'gcoder_line' extension is no longer recommended. A command such as `python setup.py build_ext --inplace` is deprecated. Therefore its outputs should no longer be ignored and should be considered cruft that needs to be removed.

Integrate all current CI actions building wheels and sdist for PyPI into a single file using `cibuildwheel`. The use of action 'RalfG/python-wheels-manylinux-build' is deprecated in favor of `PyPA/cibuildwheel`.

DivingDuck · 2024-01-26T18:31:07Z

.gitignore

@@ -10,9 +10,6 @@ prontserve-env
 pronterface.spec
 pronsole.spec
 plater.spec
-printrun/gcoder_line.c
-printrun/gcoder_line*.so
-printrun/gcoder_line*.pyd


Please do not delete as these files will be generated when compiling. This will prevent to upload those files to our repository

Fair enough, I'll revert this change.

DivingDuck · 2024-01-26T18:35:04Z

I did only a quick view and will take a closer look on this as soon I finish the translation part.

DivingDuck · 2024-01-30T19:53:00Z

@rockstorm101,
I can't see anything against implementing for windows. The protocols are looking good and the binaries are running fine on my machine so fare I can see for now.

We should ask @neofelis2X as well for a quick look.

neofelis2X · 2024-01-30T21:12:26Z

We should ask @neofelis2X as well for a quick look.

Sure, I tried the artifacts on intel and M2 Mac and they work flawlessly. Everything looks fine!

But there is indeed one tiny thing, that I can't unsee. On macos the system sees the app as 'pronterface' and not 'Pronterface' with capital 'P'. Renaming doesn't work, it seems to be the 'internal' name of the app.

PyInstaller Docs: "...the .. script .. supplies the name for the spec file and for the executable folder or file."

I believe this can be fixed by using the --name command with pyi-makespec (line 36) and changing pronterface.spec to Pronterface.spec / Pronterface.app
Sorry if this seems too nitpicky 😁 but it just looks off to me.

DivingDuck · 2024-01-31T07:27:52Z

LOL, I had never seen it written with an capital P. Looks like macOS users are the better linguists. :)

rockstorm101 · 2024-02-02T18:58:48Z

I tried the artifacts on intel and M2 Mac and they work flawlessly. Everything looks fine!

Am I right in thinking that both macOS runners produce exactly the same app? I can't test them but the sizes are identical. Same goes for the generated wheels, which seem to generate the same files at a first glance:

$ tree cibw-wheels-macos-11-2
cibw-wheels-macos-11-2
├── Printrun-2.0.1-cp310-cp310-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp310-cp310-macosx_10_9_x86_64.whl
├── Printrun-2.0.1-cp310-cp310-macosx_11_0_arm64.whl
├── Printrun-2.0.1-cp311-cp311-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp311-cp311-macosx_10_9_x86_64.whl
├── Printrun-2.0.1-cp311-cp311-macosx_11_0_arm64.whl
├── Printrun-2.0.1-cp312-cp312-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp312-cp312-macosx_10_9_x86_64.whl
├── Printrun-2.0.1-cp312-cp312-macosx_11_0_arm64.whl
├── Printrun-2.0.1-cp38-cp38-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp38-cp38-macosx_10_9_x86_64.whl
├── Printrun-2.0.1-cp38-cp38-macosx_11_0_arm64.whl
├── Printrun-2.0.1-cp39-cp39-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp39-cp39-macosx_10_9_x86_64.whl
└── Printrun-2.0.1-cp39-cp39-macosx_11_0_arm64.whl

1 directory, 15 files

$ tree cibw-wheels-macos-12-3
cibw-wheels-macos-12-3
├── Printrun-2.0.1-cp310-cp310-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp310-cp310-macosx_10_9_x86_64.whl
├── Printrun-2.0.1-cp310-cp310-macosx_11_0_arm64.whl
├── Printrun-2.0.1-cp311-cp311-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp311-cp311-macosx_10_9_x86_64.whl
├── Printrun-2.0.1-cp311-cp311-macosx_11_0_arm64.whl
├── Printrun-2.0.1-cp312-cp312-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp312-cp312-macosx_10_9_x86_64.whl
├── Printrun-2.0.1-cp312-cp312-macosx_11_0_arm64.whl
├── Printrun-2.0.1-cp38-cp38-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp38-cp38-macosx_10_9_x86_64.whl
├── Printrun-2.0.1-cp38-cp38-macosx_11_0_arm64.whl
├── Printrun-2.0.1-cp39-cp39-macosx_10_9_universal2.whl
├── Printrun-2.0.1-cp39-cp39-macosx_10_9_x86_64.whl
└── Printrun-2.0.1-cp39-cp39-macosx_11_0_arm64.whl

1 directory, 15 files

This reverts commit 837d34e.

Both macos runners produce the same wheels and are therefore redundant.

@neofelis2X

Thanks to @neofelis2X

neofelis2X · 2024-02-03T11:49:17Z

Yes they are effectively the same.
I don’t have a device with macos 11, so I can’t test if it makes a difference there specifically.

rockstorm101 · 2024-02-03T19:41:45Z

Yes they are effectively the same.

Thanks. I've removed the macOS 11 runner then.

On macos the system sees the app as 'pronterface' and not 'Pronterface' with capital 'P'. [...] I believe this can be fixed by using the --name command with pyi-makespec (line 36) and changing pronterface.spec to Pronterface.spec / Pronterface.app

Could you please give a try to the latest artifact, see if it now shows up with a capital 'P' now?

neofelis2X · 2024-02-03T19:45:45Z

Hi @rockstorm101,
already did! Perfect, the latest artifact 'printrun-test_macos-12_x64_py3.10' works. And the naming is now consistent. :D

rockstorm101 · 2024-02-10T10:24:35Z

I consider this PR merge-ready now. Should anyone present know of any reason that this PR should not be merged, speak now (or next week) or forever hold your peace.

rockstorm101 added 6 commits January 26, 2024 18:17

Add pyproject.toml and simplify Cython build

6964e11

Fixes kliment#1319

setup: Use functions for dynamic configuration

36e011a

This way no code is run when 'setup.py' is loaded until the setup function is called.

README: Update install from source guide for unix

9e87df7

Stop ignoring gcoder_line cruft

837d34e

The in-place build of the 'gcoder_line' extension is no longer recommended. A command such as `python setup.py build_ext --inplace` is deprecated. Therefore its outputs should no longer be ignored and should be considered cruft that needs to be removed.

ci: Unify all PyPI build actions

0fd23d3

Integrate all current CI actions building wheels and sdist for PyPI into a single file using `cibuildwheel`. The use of action 'RalfG/python-wheels-manylinux-build' is deprecated in favor of `PyPA/cibuildwheel`.

rockstorm101 added the WIP label Jan 26, 2024

DivingDuck reviewed Jan 26, 2024

View reviewed changes

rockstorm101 added 3 commits February 2, 2024 20:16

Revert "Stop ignoring gcoder_line cruft"

fc4db01

This reverts commit 837d34e.

ci: Remove macos-11 runner on wheel build

b88db4b

Both macos runners produce the same wheels and are therefore redundant.

ci: Produce macOS app with capital P

cfcbad5

Thanks to @neofelis2X

ci: Bump pypa/cibuildwheel from 2.16.2 to 2.16.5

436778e

rockstorm101 changed the title ~~[WIP] Add pyproject.toml and unify PyPI CI builds~~ Add pyproject.toml and unify PyPI CI builds Feb 10, 2024

rockstorm101 removed the WIP label Feb 10, 2024

DivingDuck mentioned this pull request Feb 10, 2024

Update for translation strings and translation files #1406

Merged

rockstorm101 merged commit 6b56bf8 into kliment:master Feb 27, 2024
12 checks passed

rockstorm101 deleted the pyproject-cibuild branch May 28, 2024 20:17

rockstorm101 mentioned this pull request May 29, 2024

CI actions not working as expected #1427

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pyproject.toml and unify PyPI CI builds #1407

Add pyproject.toml and unify PyPI CI builds #1407

rockstorm101 commented Jan 26, 2024

DivingDuck Jan 26, 2024

rockstorm101 Feb 1, 2024

DivingDuck commented Jan 26, 2024

DivingDuck commented Jan 30, 2024

neofelis2X commented Jan 30, 2024

DivingDuck commented Jan 31, 2024

rockstorm101 commented Feb 2, 2024

neofelis2X commented Feb 3, 2024

rockstorm101 commented Feb 3, 2024

neofelis2X commented Feb 3, 2024

rockstorm101 commented Feb 10, 2024

Add pyproject.toml and unify PyPI CI builds #1407

Add pyproject.toml and unify PyPI CI builds #1407

Conversation

rockstorm101 commented Jan 26, 2024

DivingDuck Jan 26, 2024

Choose a reason for hiding this comment

rockstorm101 Feb 1, 2024

Choose a reason for hiding this comment

DivingDuck commented Jan 26, 2024

DivingDuck commented Jan 30, 2024

neofelis2X commented Jan 30, 2024

DivingDuck commented Jan 31, 2024

rockstorm101 commented Feb 2, 2024

neofelis2X commented Feb 3, 2024

rockstorm101 commented Feb 3, 2024

neofelis2X commented Feb 3, 2024

rockstorm101 commented Feb 10, 2024