Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check benchmarks on CI #96

Merged
merged 5 commits into from May 1, 2024
Merged

Conversation

sfmig
Copy link
Contributor

@sfmig sfmig commented Apr 23, 2024

This PR follows from #94


Description

What is this PR

  • Bug fix
  • Addition of a new feature
  • Other

Why is this PR needed?
To test the benchmarks.

What does this PR do?
We test the benchmarks on CI rather than with tests.

References from other Python packages using asv to benchmark:

  • numpy: they have a separate benchmark job in Github Actions.
  • astropy: they have a cron job. Something like this is likely what we need for running the internal benchmarks.
  • scikit-image: they have a job in CI for relative benchmarking but they also run a benchmark check along the tests CI (see here).
  • napari: they also do relative benchmarks in CI, and also a quick check

Currently we are running a very basic check. We can look into further extending this with maybe something closer to the napari or numpy approaches (see below).

References

This PR follows from #94 and relates to issue #9. It should be rebased to main after #94 is squash-and-merged.

How has this PR been tested?

Please explain how any new code has been tested, and how you have ensured that no existing functionality has changed.

Is this a breaking change?

If this PR breaks any existing functionality, please explain how and why.

Does this PR require an update to the documentation?

If any features have changed, or have been added. Please explain how the documentation has been updated (and link to the associated PR). See here for details.

Checklist:

  • The code has been tested locally
  • Tests have been added to cover all new functionality (unit & integration)
  • The documentation has been updated to reflect any changes
  • The code has been formatted with pre-commit

@sfmig sfmig force-pushed the smg/move-benchmark-testing-to-ci branch from 1c3dacc to 10ce5ab Compare April 23, 2024 10:51
Copy link

codecov bot commented Apr 23, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.69%. Comparing base (34b07ec) to head (e097500).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #96      +/-   ##
==========================================
+ Coverage   84.45%   85.69%   +1.24%     
==========================================
  Files          17       16       -1     
  Lines         862      839      -23     
==========================================
- Hits          728      719       -9     
+ Misses        134      120      -14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sfmig sfmig force-pushed the smg/move-benchmark-testing-to-ci branch from edd00b6 to 28ffd78 Compare April 25, 2024 10:47
@sfmig sfmig marked this pull request as ready for review April 25, 2024 16:40
@sfmig sfmig marked this pull request as draft April 25, 2024 16:40
@sfmig sfmig force-pushed the smg/move-benchmark-testing-to-ci branch from 28ffd78 to e097500 Compare April 30, 2024 14:44
@sfmig sfmig marked this pull request as ready for review April 30, 2024 16:13
@sfmig sfmig requested a review from a team April 30, 2024 16:13
@sfmig sfmig mentioned this pull request Apr 30, 2024
7 tasks
Copy link
Member

@alessandrofelder alessandrofelder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thank you!

@alessandrofelder alessandrofelder merged commit e175ec8 into main May 1, 2024
12 checks passed
@alessandrofelder alessandrofelder deleted the smg/move-benchmark-testing-to-ci branch May 1, 2024 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants