Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a workflow to release and sign wheels #22

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

agriyakhetarpal
Copy link
Contributor

Description

Partially addresses issues raised in #20. This PR adds a workflow to release wheels to PyPI and sign them.

These changes are in response to the comment:

I think we can start with a workflow responding to workflow_dispatch event where the workflow itself uses both Sigstore and Trusted Publishing, with the pushing of wheels still being manual. Then we can consider something more automated after that.

and the workflow is meant to be triggered manually.

Changes made

  1. A workflow that responds to the manual workflow_dispatch: trigger, with customisable inputs for enabling the version to build wheels for, the suffix to append to the version, platforms to build wheels for (might be redundant), and lastly, a push_to_pypi input which is false by default, but can be set to true to trigger the deployment job.
  2. The deployment job is separate from the job that builds wheels because to isolate the environment associated with the job, and the extra permissions that it requires. Artifacts (in this case, wheels) are shared between jobs.
  3. The deployment job signs the wheels and generates .sigstore files, which can be used to verify that the wheels were indeed generated inside a GitHub Actions workflow.

Additional context

To further mitigate actors bearing malicious intent, I would recommend adding a CODEOWNERS file or some sort of branch protection so that it is not easy for others to modify the workflow and trigger it. Currently, those with triage access can trigger workflows. The environment with the name pypi has to be set up and requests to use it to deploy have to be manually approved by a repository administrator or someone with the appropriate permissions, so, I would recommend keeping that stringent

Comment on lines +15 to +19
platforms:
description: >
Comma-separated values of platforms to build wheels for
required: false
default: 'x86_64-windows,x86-windows,x86_64-macos,aarch64-macos,i386-linux,x86-linux,x86_64-linux,aarch64-linux,armv7a-linux'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be overkill as well – I was just experimenting with this locally, so it can be good if you don't wish to build too many wheels at once, but if a wheel or architecture has to be backported for a particular version, it can be useful.

push_to_pypi:
description: >
Whether to push the built wheels to PyPI. Can be 'true' or 'false', defaults to 'false'.
required: false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
required: false
required: true

It might make sense to make this a required input so that the user is aware of what is being input here.

Comment on lines +60 to +62
needs: [build_wheels]
environment: pypi
runs-on: ubuntu-latest
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This environment will have to be created in the repository settings, as noted in the PR description. It will trigger the workflow, but the deployment will pass only if it gets approved, and the workflow won't be triggered until so.

Comment on lines +84 to +95
- name: Sign artifacts with Sigstore
uses: sigstore/gh-action-sigstore-python@61f6a500bbfdd9a2a339cf033e5421951fbc1cd2 # v2.1.1
with:
inputs: >-
./dist/*.whl

- name: Upload signed artifacts and signature files
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
# This will contain not only the wheels but also the signature files
# generated by the Sigstore step
path: dist/*
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The artifact retention policy for GitHub Actions artifacts, which are, in this case, the files generated from Sigstore, is set to default to 90 days from the completion of the workflow (not sure what the upper limit is). Having a place to put them permanently would serve some part of the reproducibility issues discussed. The available options are:

  1. Publish them to GitHub Releases: and releases require git tags, which is something that was objected to and we agreed not to do
  2. Upload them in a folder in the repository in a manual commit after downloading them (not the wheels though, of course)
  3. Upload them to a pinned issue and store them in the issue comments
  4. and so on.

I think the second one is the best option at this time, because users can download and use those to verify them.

Comment on lines 68 to 71
if: >-
github.event_name == 'workflow_dispatch' &&
github.event.inputs.push_to_pypi == 'true' &&
github.repository == 'ziglang/zig-pypi'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll move this to the context of the step that uploads to PyPI instead of the entire job, because we can have scenarios where we want to build a wheel, upload to PyPI manually (locally) using twine, but also have the Sigstore files associated with it as marker that the upload has been signed.

This can be useful in cases like #18 where PyPI already has Zig wheels for some platforms for a particular version, and one needs to build extra wheels for two versions, so instead of doing so locally, one can do it from here by triggering the workflow twice, and push the Sigstore files somewhere later to establish their permanence.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in cf9845e.

@agriyakhetarpal
Copy link
Contributor Author

Here's a brief implementation as requested, @whitequark – would appreciate general comments based on how you would like to refine it and whether it fits with the ideas that you had in mind :)

@whitequark
Copy link
Collaborator

Thanks, impressive work! I won't have time to review it until a week or two from now most likely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants