Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore CI benchmarks #933

Open
Robbepop opened this issue Feb 8, 2024 · 1 comment
Open

Restore CI benchmarks #933

Robbepop opened this issue Feb 8, 2024 · 1 comment
Labels
ci CI work item help wanted Extra attention is needed

Comments

@Robbepop
Copy link
Collaborator

Robbepop commented Feb 8, 2024

With the transition of Wasmi from the paritytech GitHub organization to the wasmi-labs GitHub organization we also (expectedly) lost the GitLab based CI benchmarking for Wasmi which immensely helped the Wasmi developers keep track of the performance implications of their PRs.

This was done by automated benchmarks for each commit of a PR to master and a comment that summarizes the results in detail, such as this: #928 (comment)

Unfortunately with the transition we no longer have access to Parity's GitLab instance and runners and our current CI setup no longer works. Affected files are:

Ideally the new CI uses GitHub Actions and self-hosted runners compatible with GitHub actions that are similar in hardware specs (or better) than the original ones we had with our GitLab instance. For that purpose we might want to rent some proper hardware to do the job in the future. But for experimental purposes it should be fine to build initial GitHub Actions based CI benchmarks with default GitHub action runners.

So far we did the parsing of the benchmark results with the benchmark-report.sh file but maybe we could use this crate and a proper internal Rust crate for the job in the future: https://crates.io/crates/criterion-table

cc @alvicsam @sergejparity

@Robbepop Robbepop added help wanted Extra attention is needed ci CI work item labels Feb 8, 2024
@Robbepop
Copy link
Collaborator Author

Removed GitLab benchmarking CI related files here: #951

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci CI work item help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant