Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache build steps and artifacts #3096

Merged
merged 4 commits into from May 14, 2024
Merged

Cache build steps and artifacts #3096

merged 4 commits into from May 14, 2024

Conversation

Jakuje
Copy link
Member

@Jakuje Jakuje commented Apr 3, 2024

This is an attempt to speed up the build times and reduce the resources by caching where possible:

  • Letting github to install java for use and cache Maven dependencies (which take ages to download and spam the log)
  • Using github action to cache APT dependencies -- they are usually the ones that take ages to download and install and most of them are same in each of the runs
  • Caching the libressl build between different runs (which is still the same version for each PR and build)
  • Skipping the clang-tidy from the first build pipeline as it takes ages and somehow got installed by default for some reason.

Its cumbersome at times and there is probably some place for improvement, but I think it should do the job. For comparsion, last builds on master:

The setup is now much faster:

The builds and tests still take considerable amount of time though.

@Jakuje Jakuje force-pushed the cache-build branch 4 times, most recently from 88d41e5 to 7ec8d0f Compare April 3, 2024 11:46
@frankmorgner
Copy link
Member

Thanks for thinking of the environment 😉

Maybe we could also use autoconf's caching, although I'm not sure if that works when we're doing autoreconf in between the new and the cached run of ./configure -C.

@Jakuje
Copy link
Member Author

Jakuje commented Apr 3, 2024

Maybe we could also use autoconf's caching, although I'm not sure if that works when we're doing autoreconf in between the new and the cached run of ./configure -C.

If we would like to share this among the PRs, we would have to figure out what all comes into the play of creating them (configure, Makefiles?) to base the cache on them. I can certainly try that if it will work with different suffixes to bootstrap PRs.

I just tried to see what parts of build take how much time, but ./bootstrap is roughly 6s in fresh clone in container, configure (without arguments) 6s, and make (4 threads) 35s (or 2 mins single-threaded -- not sure how many cores the GH runners have) so this should be quite small amount of time saved.

@frankmorgner
Copy link
Member

Agreed, then it doesn't make sense to have a deeper look into this.

An other option would be to use ccache to speed up the actual build. We could use this GH action or use something like ./configure CC="ccache gcc" and cache CCACHE_DIR ourselves (see https://ccache.dev/manual/4.9.1.html)

Copy link
Member

@frankmorgner frankmorgner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ccache may be an improvement for a future PR.

@Jakuje
Copy link
Member Author

Jakuje commented May 14, 2024

I was hoping I will be able to get to look into the ccache over the last month, but it does not look like the case so I will merge the current version and open a new issue for the ccache usage.

@Jakuje Jakuje merged commit 4fa7825 into OpenSC:master May 14, 2024
42 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants