Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find a way to avoid redundant CI workflow builds of cpython just to test Docs #435

Open
ned-deily opened this issue Mar 28, 2022 · 4 comments

Comments

@ned-deily
Copy link
Member

Currently the GitHub CI workflow for doc seems to first do its own build of cpython before attempting to build the docs. While that may be necessary for correct running of doctests or other reasons, it seems wasteful and time-consuming when cpython is already being built in other CI workflows. It would be great if someone could find a way to avoid unnecessary builds by perhaps combining the workflows in some way. See .github/workflows in the cpython repo.

@brandtbucher
Copy link
Member

Unformed idea: I believe that...

  • Actions can share "artifacts" with each other.
  • Actions can depend on other actions.

Perhaps we could have a "build Python" action that provides a binary artifact to both the Docs and Ubuntu jobs?

@hugovk
Copy link
Member

hugovk commented Mar 28, 2022

Yes, there could be a build job that then uploads the build artifact using https://github.com/actions/upload-artifact and then two dependant jobs that use https://github.com/actions/download-artifact to fetch it and do their respective tests.

For:

  • Just one CPython build
  • Main and docs test can run in parallel
  • And independently

Against:

  • Need to check if there are limits on total size of artifacts stored. In any case we don't need to keep them longer for a day, default is something like 90 days.
  • Bit of extra time for starting up new jobs

Couple of other ideas:

The build could be moved inside the main workflow as a new step after the main build/tests. Can still be run only when docs files change.

For:

  • No need to build twice

Against:

  • Doesn't run in parallel
  • If main tests fail, docs build/test won't run
  • Makes the main workflow longer

While that may be necessary for correct running of doctests or other reasons, ...

Alternatively, that "may" suggests maybe we don't need to build Python from source. If not, we can use a GitHub Actions version:

For:

  • Much faster to install
  • Can keep parallel builds
  • Can test docs even if main tests fail
  • Can use a GitHub Actions 3.11-dev to test on latest alpha/beta/RC
  • Or can use deadsnakes to test on a nightly build

Against:

  • Not testing against latest main or PR branch

@arhadthedev
Copy link
Member

arhadthedev commented Mar 28, 2022

Need to check if there are limits on total size of artifacts stored. In any case we don't need to keep them longer for a day

Maybe, actions/cache then (docs)? It gives 10GB per repo with LRU autoeviction.

Also it works way faster than upload/download-artifact beause the latter packs/unpacks everything into a zip archive to make it downloadable not only by actions but by users too.

@hugovk
Copy link
Member

hugovk commented Jun 7, 2022

(See also #459.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants