Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] feat: mark and skip flaky test #746

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

marco-ippolito
Copy link
Member

@marco-ippolito marco-ippolito commented Oct 18, 2023

Draft implementation to add flaky tests automatically to the .status file.
Missing:

  • rules to determine which tests we consider flaky
  • tests
  • proper matching for OS and architecture

I'm assuming that if the failed test shows up in at least 5 pr is flaky (debatable) and can be skipped, I'd increase this number to make sure no false positive

@codecov
Copy link

codecov bot commented Oct 20, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6d68c99) 83.03% compared to head (1a12479) 82.95%.
Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #746      +/-   ##
==========================================
- Coverage   83.03%   82.95%   -0.08%     
==========================================
  Files          37       37              
  Lines        4178     4190      +12     
==========================================
+ Hits         3469     3476       +7     
- Misses        709      714       +5     

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant