Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: align test save location configs #3585

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

yangannyx
Copy link
Contributor

@yangannyx yangannyx commented Apr 30, 2024

  • I have read the contribution documentation for this project.
  • I agree to follow the code of conduct that this project follows, as appropriate.
  • The changes are appropriately documented (if applicable).
  • The changes have sufficient test coverage (if applicable).
  • The testsuite passes successfully on my local machine (if applicable).

Summarize your changes:
CircleCI can't detect previous test timing data. Fixing this by tweaking the script we use to ensure filepaths in the junit xml files are using relative paths
Testing:

  • Tested that running the new script for making test paths relative works on a test xml file generated locally using CI=1 yarn test --suite="slow"
  • Checking CircleCI Run Slow Tests output to ensure that circle was able to autodetect file timings
Screenshot 2024-04-30 at 1 53 59 PM

@yangannyx yangannyx requested a review from a team as a code owner April 30, 2024 19:24
Copy link
Member

@dsanders11 dsanders11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it's still having trouble with Windows.

@DevanceJ
Copy link
Contributor

DevanceJ commented May 1, 2024

Thank you for this!

@erickzhao erickzhao mentioned this pull request May 14, 2024
5 tasks
@yangannyx
Copy link
Contributor Author

I just merged main with no additional changes to the previous, and surprisingly filename timings across all platforms were detected. This makes me think that the timing detection issue we saw on the previous slow Windows CI run wasn't caused by outputting OS incompatible relative paths. My theory right now is that the path detection relies on testing output from previous circle runs. I wish I could ssh into the circle testing machines to confirm the content of the test folder and get a better mental model, but it looks like Rerun with SSH is disabled.

I want to confirm the following:

  1. Is filename timing detection broken on main? To test this I will 1a (commit). comment out the current logic changes to generating the relative paths and set a new folder path for CI runs to store test data in. This will hopefully prevent interference from other CI builds on other PRs. Then I will 1b (commit). make an inconsequential change and see if the test output from the previous commit detects filename timing. If commit 1b succeeds, I don't think there's an issue on main.
  2. If commit 1b fails, I will retry 1a and 1b but with the relative path replacement changes I made. I'll see where this takes me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants