Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ez] Add --save-xml option to run_test #126690

Closed
wants to merge 3 commits into from
Closed

Conversation

clee2000
Copy link
Contributor

@clee2000 clee2000 commented May 20, 2024

Move configuration of xml generation from checking for IS_CI to using an argument flag

Remove the parser from common_utils as a parent parser, I don't think any of those options are being used in run_test.py, so if people do want to use them, they can be added using the additional_unittest_args option

Copy link

pytorch-bot bot commented May 20, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126690

Note: Links to docs will display an error until the docs builds have been completed.

❌ 32 New Failures, 1 Unrelated Failure

As of commit 362403f with merge base 3642e51 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label May 20, 2024
@clee2000 clee2000 changed the title Add --save-xml option to run_test [ez] Add --save-xml option to run_test May 20, 2024
@clee2000 clee2000 marked this pull request as ready for review May 20, 2024 17:14
@clee2000 clee2000 requested a review from a team as a code owner May 20, 2024 17:14
@@ -1257,6 +1255,12 @@ def parse_args():
action="store_false",
help="Run tests without translation validation.",
)
parser.add_argument(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work as expected: common_util parses the cmdline arguments on import and sets variables accordingly.

Some are used by get_report_path which would be broken afterwards:

TEST_SAVE_XML = args.save_xml

test_report_path = TEST_SAVE_XML + LOG_SUFFIX

I'm a bit confused about the run_test function in common_utils and how/where it is used but that would already pass the --save-xml flag on:

if TEST_SAVE_XML:
other_args += ['--save-xml', args.save_xml]

Maybe the 2 functions (in run_test and common_utils) can be merged?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to generate xml to prevent clashing, I'll probably get rid of or change the save xml option in common_utils once I figure out what parts of CI it's affecting

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the parents=[common_parser], means that --save-xml is no longer recognized and will throw an error when used. It likely affects much more from that file.
Possibly it is easier just to change https://github.com/pytorch/pytorch/pull/126690/files#diff-e72503c9e3e8766e2d1bacf3fad7b88aa166e0e90a7e103e7df99357a35df8d7R992 to if IS_CI or options.save_xml: and make sure --save-xml is passed to all subprocesses.

@clee2000
Copy link
Contributor Author

Closing this, I misread the code so my understanding was wrong

@clee2000 clee2000 closed this May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants