Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try pytest-xdist and pytest-rerunfailures #229

Closed
wants to merge 8 commits into from

Conversation

mscheltienne
Copy link
Member

@mscheltienne mscheltienne commented Mar 14, 2024

Closes #228

Locally, test run fine and rarely require retries.
On CIs, some tests are flakky. Let's mark those with pytest-rerunfailures.
But to do so, those test need to be run by workers through xdist to recover from e.g. segfaults.
But all tests can not be run through xdist, e.g. stream resolution.

Plan:

  • Define pytest.mark.no_xdist for serial execution
  • Mark test which should not be run in parallel
  • In CIs, edit the pytest step to run once the test which are not marked with no_xdist with the arguments -n logical --dist worksteal
  • In CIs, edit the pytest step to run once the test which are marked with no_xdist

Deprecation in pytest-cov due to pytest-dev/pytest-cov#623

@mscheltienne mscheltienne added this to the 1.3 milestone Mar 14, 2024
@mscheltienne mscheltienne self-assigned this Mar 14, 2024
@mscheltienne mscheltienne removed this from the 1.3 milestone Mar 15, 2024
@mscheltienne mscheltienne marked this pull request as draft March 15, 2024 09:36
@mscheltienne
Copy link
Member Author

For now, not worth the implementation work.

@mscheltienne mscheltienne deleted the rerun branch March 18, 2024 12:16
@mscheltienne
Copy link
Member Author

Some of the test changes in this PR have been merged through #231

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Try pytest-rerunfailures instead of nick-fields/retry@v3 to reduce CI time and feedback upon failure
1 participant