Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running pytests in parallel #363

Draft
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

jeremyleung521
Copy link
Member

Issue Number. Is this pull request related to any outstanding issues? If so, list the issue number.

Describe the changes made. A clear and concise description of what the problem is and what you did to fix it. E.g. [...] was happening and I've changed [...] to fix it.
pytest-xdist is a plugin that allows us to run our pytests in parallel. For example, MacOS runners on GitHub Actions have 4 workers, but isn't being taken advantaged of.

Tests are distributed among workers randomly, so there might be more instances of errors when tests are executed out-of-order.

Examples of success: https://github.com/jeremyleung521/westpa/actions/runs/7012321093/job/19111251133
Linux tests that used to take 4 minutes now take 1.5 minutes.

Goals and Outstanding Issues. A clear and concise list of goals (to be) accomplished.

  • Speed up GitHub Action tests
  • Tests will continue to run even if it fails on a single configuration

Major files changed.

  • .github/workflows/test.yaml
  • devtools/conda-envs/test_env.yaml
  • setup.py

Status.

  • New feature (non-breaking change which adds functionality)
  • I have read the CONTRIBUTING document.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Additional context. Add any other context or screenshots about the pull request here.

@jeremyleung521 jeremyleung521 added the CI maintenance For PRs and issues that improve/maintain CI workflows label Nov 28, 2023
Copy link

codecov bot commented Nov 28, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (4934c21) 41.89% compared to head (6138859) 41.84%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #363      +/-   ##
===========================================
- Coverage    41.89%   41.84%   -0.05%     
===========================================
  Files          131      131              
  Lines        17011    17011              
===========================================
- Hits          7127     7119       -8     
- Misses        9884     9892       +8     
Flag Coverage Δ
unittests 41.84% <ø> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jeremyleung521
Copy link
Member Author

Results are inconsistent so might have to implement our own scheduler. Will turn this into draft for now.

Some resources:
pytest-dev/pytest-xdist#18
https://stackoverflow.com/questions/59498774/how-to-tell-pytest-xdist-to-run-tests-from-one-folder-sequencially-and-the-rest

@jeremyleung521 jeremyleung521 marked this pull request as draft November 28, 2023 20:59
@jeremyleung521 jeremyleung521 self-assigned this Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI maintenance For PRs and issues that improve/maintain CI workflows
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants