Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce time for running tests. #166

Closed
MinoruSekine opened this issue Nov 8, 2020 · 6 comments
Closed

Reduce time for running tests. #166

MinoruSekine opened this issue Nov 8, 2020 · 6 comments
Assignees
Labels
good first issue Good for newcomers
Projects
Milestone

Comments

@MinoruSekine
Copy link
Owner

MinoruSekine commented Nov 8, 2020

Is your feature request related to a problem? Please describe.

  • There are 4000 numbers of tests or more, and it takes 1 miniutes or more to run on GitHub Actions.
    Sometimes it interrupts our works.

Describe the solution you'd like

  • Reduce range of parameterized tests to less enough

Describe alternatives you've considered

  • Remove slow GitHub Actions environments from check build

Additional context

  • N/A
@MinoruSekine MinoruSekine added the good first issue Good for newcomers label Nov 8, 2020
@MinoruSekine MinoruSekine added this to the Backlog milestone Nov 8, 2020
@MinoruSekine MinoruSekine self-assigned this Nov 8, 2020
@MinoruSekine MinoruSekine added this to To do in Features via automation Nov 8, 2020
@MinoruSekine MinoruSekine moved this from To do to In progress in Features Dec 17, 2020
@MinoruSekine
Copy link
Owner Author

In latest check build,

OS Build type Duration(before)
Ubuntu-latest release 1m19s
Ubuntu-latest debug 50s
macOS-latest release 2m11s
macOS-latest debug 1m09s
windows-latest,x64 release 1m46s
windows-latest,x64 debug 1m18s
windows-latest,x86 release 1m46s
windows-latest,x86 debug 1m18s

@MinoruSekine
Copy link
Owner Author

I wonder why release build takes longer duration than debug.
In my little investigation, it seems build time for release is much longer than debug.
And most part of total duration is build time.

@MinoruSekine
Copy link
Owner Author

(Before)

[----------] Global test environment tear-down
[==========] 4649 tests from 140 test suites ran. (40604 ms total)
[  PASSED  ] 4649 tests.

@MinoruSekine
Copy link
Owner Author

(after)

[----------] Global test environment tear-down
[==========] 1861 tests from 140 test suites ran. (17566 ms total)
[  PASSED  ] 1861 tests.

@MinoruSekine
Copy link
Owner Author

OS Build type Duration(before) Duration(After)
Ubuntu-latest release 1m19s 1m12s
Ubuntu-latest debug 50s 56s
macOS-latest release 2m11s 2m28s
macOS-latest debug 1m09s 1m15s
windows-latest,x64 release 1m46s 1m55s
windows-latest,x64 debug 1m18s 1m18s
windows-latest,x86 release 1m46s 1m43s
windows-latest,x86 debug 1m18s 1m9s

Duration on macOS can be affected by actions/runner-images#2247 .

But, gmmm...

@MinoruSekine
Copy link
Owner Author

OS Build type Duration(before) Duration(After) Duration(Latest CI)
Ubuntu-latest release 1m19s 1m12s 1m17s
Ubuntu-latest debug 50s 56s 56s
macOS-latest release 2m11s 2m28s 1m26s
macOS-latest debug 1m09s 1m15s 1m30s
windows-latest,x64 release 1m46s 1m55s 1m34s
windows-latest,x64 debug 1m18s 1m18s 1m10s
windows-latest,x86 release 1m46s 1m43s 1m35s
windows-latest,x86 debug 1m18s 1m9s 1m9s

Not enough tested, but it seems there are no serious bad side effects and there are some small improvements.
Now this ticket should be closed.
It will be re-opened or created as a new one if duration for CI will be felt wrong.

Features automation moved this from In progress to Done Dec 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
Features
  
Done
Development

No branches or pull requests

1 participant