Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added option to only generate HTML/JUnit for failed tests via -f #236

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

amleszk
Copy link
Contributor

@amleszk amleszk commented Nov 5, 2021

At Grubhub there are CI builds where we run the XCHTMLReport against 14000 unit tests, which takes a long time because the app makes repeated calls to the terminal app invoked by external library .package(url: "https://github.com/davidahouse/XCResultKit.git", .upToNextMinor(from: "0.9.2")), .
Exact source: https://github.com/davidahouse/XCResultKit/blob/main/Sources/XCResultKit/XCResultFile.swift#L136

And the build time for this report takes 30 minutes, which is way too long.

This PR makes a high level change to introduce a new parameter -f which will ignore reporting on tests that have passed. Only reporting on failures. This reduces the report build time to about 1 minute, and still allows for the report to contain the information that is relevant to a CI build failure.

image

Changes

  • Refactored rendering model to take RenderingArguments
  • Added new option -f
  • Added isEmpty to and compactMap Test models to filter those that are empty of any tests. removing noise from the test report
  • Added filtering in if status == .success && renderingArgs.failingTestsOnly {

Copy link
Member

@tylervick tylervick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice addition!

@amleszk Are you able to share any additional details on the time complexity you're seeing?
I'm wondering if there's anything we can do to reduce extraneous calls made to xcrun, particularly around reusing extracted data from actions, etc.

@kronenthaler
Copy link

+1 for this

@amleszk
Copy link
Contributor Author

amleszk commented Dec 18, 2021

Thanks @tylervick i think the only fix would be to stop depending on XCResultKit or refactor some sort of parallel report generation architecture in XCHTMLReport. but also is that worth the effort? I would recommend not having 14,000 unit tests in one Xcode target anyway and instead splitting those tests into sub-targets. Which is exactly what is the new approach we're doing here :)

@amleszk
Copy link
Contributor Author

amleszk commented Dec 18, 2021

I still cannot merge this pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants