Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable ccache / sccache on our Github CI Actions #436

Open
gpshead opened this issue Mar 28, 2022 · 5 comments
Open

Enable ccache / sccache on our Github CI Actions #436

gpshead opened this issue Mar 28, 2022 · 5 comments

Comments

@gpshead
Copy link
Member

gpshead commented Mar 28, 2022

This should significantly reduce build latency, particularly rebuild latency when minor changes are pushed in PR branches.

It can help across builds, but as the main branch point of a given PR branch is not synchronized how much it helps there will be limited. A lot more likely to help on release branches for bugfix backports as C code in those trees doesn't evolve quickly.

https://github.com/hendrikmuhs/ccache-action has instructions. sccache even claims to support Windows (I've never used it).

@ammaraskar
Copy link
Member

Looks like we even have the ccache action already hovering around in the workflow from here: https://github.com/python/cpython/pull/29935/files

Just need to use it for the C building bits.

@ammaraskar
Copy link
Member

Actually, since the workflow uses the approach that adds the ccache folder to PATH:

https://github.com/python/cpython/blob/788154919c2d843a0a995994bf2aed2d074761ec/.github/workflows/build.yml#L67-L68

      - name: Add ccache to PATH
        run: echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV

we might already be using ccache for all the subsequent steps including Build CPython.

@ammaraskar
Copy link
Member

Yeah, we definitely already are thanks to @tiran. Here's a comparison of the same commit built on python/cpython with a full cache vs my personal fork with no cache:

python/cpython

original build time

ammaraskar/cpython

fork build time


I think the action items here are adding the ccache action to the docs and mac build to speed them up and maybe looking at swapping out for sccache so it can be used on Windows as well.

@gpshead
Copy link
Member Author

gpshead commented Mar 29, 2022

It also looked like sccache can be used with Windows if someone win-familiar could figure out how that needs to integrate with its build system.

@erlend-aasland
Copy link

We've been using ccache in our CI for a while. Should we keep this open for the Windows CI?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants