Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[20843] Detect flaky tests Github workflow #4752

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

elianalf
Copy link
Contributor

@elianalf elianalf commented May 3, 2024

Description

This PRs adds a workflow to detect flaky tests.
It adds a nightly workflow:

  • run the tests and save the results in a junit format
  • download the previous results saved in artifacts
  • detect the flaky tests
  • upload the new test result in the artifact

The detection of flaky tests is executed through a python script:

  • detect the flaky tests and compute the flakiness score
  • summarize the output in the summary of the job

Contributor Checklist

  • Commit messages follow the project guidelines.
  • The code follows the style guidelines of this project.
  • N/A Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally
  • N/A Any new/modified methods have been properly documented using Doxygen.
  • N/A Any new configuration API has an equivalent XML API (with the corresponding XSD extension)
  • Changes are ABI compatible.
  • Changes are API compatible.
  • N/A New feature has been added to the versions.md file (if applicable).
  • N/A New feature has been documented/Current behavior is correctly described in the documentation.
  • N/A Applicable backports have been included in the description.

Reviewer Checklist

  • The PR has a milestone assigned.
  • The title and description correctly express the PR's purpose.
  • Check contributor checklist is correct.
  • Check CI results: changes do not issue any warning.
  • Check CI results: failing tests are unrelated with the changes.

@elianalf elianalf added this to the v3.0.0 milestone May 3, 2024
@EduPonz EduPonz force-pushed the 3.0.x-devel branch 2 times, most recently from 08ee62b to 623db82 Compare May 13, 2024 09:26
@EduPonz EduPonz deleted the branch master May 14, 2024 07:22
@EduPonz EduPonz closed this May 14, 2024
@EduPonz EduPonz reopened this May 14, 2024
@EduPonz EduPonz changed the base branch from 3.0.x-devel to master May 14, 2024 07:26
@elianalf elianalf added needs-review PR that is ready to be reviewed needs rebase labels May 14, 2024
@elianalf elianalf force-pushed the feature/detect_flaky_tests branch from 3f5e774 to adb0412 Compare May 31, 2024 07:24
…ness

Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
…nalysis

Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
…en will get the last available artifact from the previous workflow

Signed-off-by: elianalf <62831776+elianalf@users.noreply.github.com>
@elianalf elianalf force-pushed the feature/detect_flaky_tests branch 2 times, most recently from 1f7e650 to c9b82c1 Compare June 4, 2024 06:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-review PR that is ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants