Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(cli): -p/ --parallel option defines tests amount to be run on parallel #6381

Merged
merged 20 commits into from May 15, 2024
Merged

feat(cli): -p/ --parallel option defines tests amount to be run on parallel #6381

merged 20 commits into from May 15, 2024

Conversation

tsuf239
Copy link
Collaborator

@tsuf239 tsuf239 commented Apr 30, 2024 •
edited

Adding -b/ --batch option defines tests amount to be run on parallel,
as well as renaming the sdk spec tests file
potentially fixes #6363

Checklist

  • Title matches Winglang's style guide
  • Description explains motivation and solution
  • Tests added (always)
  • Docs updated (only required for features)
  • Added pr/e2e-full label if this feature requires end-to-end testing

By submitting this pull request, I confirm that my contribution is made under the terms of the Wing Cloud Contribution License.

@tsuf239 tsuf239 requested a review from a team as a code owner April 30, 2024 12:23
@github-actions GitHub Actions
Copy link

Thanks for opening this pull request! 🎉
Please consult the contributing guidelines for details on how to contribute to this project.
If you need any assistence, don't hesitate to ping the relevant owner over Slack.

Topic Owner
Wing SDK and utility APIs @chriscbr
Wing Console @ainvoner, @skyrpex, @polamoros
JSON, structs, primitives and collections @hasanaburayyan
Platforms and plugins @hasanaburayyan
Frontend resources (website, react, etc) @tsuf239
Language design @chriscbr
VSCode extension and language server @markmcculloh
Compiler architecture, inflights, lifting @yoav-steinberg
Wing Testing Framework @tsuf239
Wing CLI @markmcculloh
Build system, dev environment, releases @markmcculloh
Library Ecosystem @chriscbr
Documentation @hasanaburayyan
SDK test suite @tsuf239
Examples @hasanaburayyan
Wing Playground @eladcon

@monadabot
Copy link
Contributor

monadabot commented Apr 30, 2024 •
edited

Console preview environment is available at https://wing-console-pr-6381.fly.dev 🚀

Last Updated (UTC) 2024-05-15 07:33

eladb
eladb reviewed May 1, 2024
View reviewed changes
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the PR title and description according to the guidelines and fix capitalization.

apps/wing/src/cli.ts Outdated Show resolved Hide resolved
@tsuf239 tsuf239 requested a review from a team May 2, 2024 08:28
@tsuf239 tsuf239 changed the title feat(cli): adding -b/ --batch option to define how many tests will run on parallel feat(cli): -b/ --batch option defines tests amount to be run on parallel May 2, 2024
MarkMcCulloh
MarkMcCulloh requested changes May 2, 2024
View reviewed changes
apps/wing/src/cli.ts Outdated Show resolved Hide resolved
@tsuf239 tsuf239 changed the title feat(cli): -b/ --batch option defines tests amount to be run on parallel feat(cli): -p/ --parallel option defines tests amount to be run on parallel May 5, 2024
@tsuf239 tsuf239 requested a review from MarkMcCulloh May 5, 2024 11:46
MarkMcCulloh
MarkMcCulloh reviewed May 7, 2024
View reviewed changes
apps/wing/src/commands/test/test.ts Outdated Show resolved Hide resolved
apps/wing/src/commands/test/test.ts Outdated Show resolved Hide resolved
apps/wing/package.json Outdated Show resolved Hide resolved
tsuf239 and others added 6 commits May 8, 2024 16:14
@tsuf239 @MarkMcCulloh
Update apps/wing/package.json
da27e11 
Co-authored-by: Mark McCulloh <Mark.McCulloh@gmail.com>
@tsuf239 @MarkMcCulloh
Update apps/wing/src/commands/test/test.ts
f0a5a1f 
Co-authored-by: Mark McCulloh <Mark.McCulloh@gmail.com>
@tsuf239
Merge branch 'main' into tsuf/run-tests-in-batches
4fc39db
@tsuf239 @MarkMcCulloh
Update apps/wing/src/commands/test/test.ts
6dcf8a2 
Co-authored-by: Mark McCulloh <Mark.McCulloh@gmail.com>
@monadabot
Merge branch 'main' into tsuf/run-tests-in-batches
bfd8e71
@monadabot
chore: self mutation (build.diff)
f56a0ae 
Signed-off-by: monada-bot[bot] <monabot@monada.co>
@monadabot monadabot added the ⚠️ pr/review-mutation PR has been mutated and will not auto-merge. Clear this label if the changes look good! label May 8, 2024
MarkMcCulloh
MarkMcCulloh reviewed May 8, 2024
View reviewed changes
apps/wing/src/cli.ts Outdated Show resolved Hide resolved
apps/wing/src/commands/test/test.test.ts Outdated Show resolved Hide resolved
MarkMcCulloh
MarkMcCulloh reviewed May 9, 2024
View reviewed changes
apps/wing/src/cli.ts Outdated Show resolved Hide resolved
@tsuf239 tsuf239 removed the ⚠️ pr/review-mutation PR has been mutated and will not auto-merge. Clear this label if the changes look good! label May 12, 2024
@tsuf239 tsuf239 requested a review from MarkMcCulloh May 12, 2024 13:41
@winglang winglang deleted a comment from monadabot May 13, 2024
MarkMcCulloh
MarkMcCulloh approved these changes May 13, 2024
View reviewed changes
Copy link
Contributor

@MarkMcCulloh MarkMcCulloh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

apps/wing/src/commands/test/test.test.ts Show resolved Hide resolved
apps/wing/src/commands/test/test.ts Show resolved Hide resolved
tsuf239 and others added 2 commits May 15, 2024 10:24
@tsuf239 @MarkMcCulloh
Update apps/wing/src/commands/test/test.ts
57015e4 
Co-authored-by: Mark McCulloh <Mark.McCulloh@gmail.com>
@tsuf239
Revert "Update apps/wing/src/commands/test/test.ts"
d187fa3 
This reverts commit 57015e4.
@monadabot
Copy link
Contributor

Benchmarks

Comparison to Baseline 🟥⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜
Benchmark Before After Change
version 55ms±0.39 59ms±3.63 +4ms (+6.45%)🟥
functions_1.test.w -t sim 410ms±22.56 401ms±16.46 -8ms (-2.07%)⬜
functions_1.test.w -t tf-aws 821ms±5.6 828ms±7.62 +7ms (+0.88%)⬜
jsii_big.test.w -t sim 2784ms±7.42 2800ms±19.16 +16ms (+0.56%)⬜
jsii_big.test.w -t tf-aws 2993ms±10.95 3001ms±12.59 +8ms (+0.27%)⬜
functions_10.test.w -t sim 486ms±6.2 494ms±9.58 +8ms (+1.58%)⬜
functions_10.test.w -t tf-aws 2006ms±10.67 2014ms±14.5 +9ms (+0.43%)⬜
jsii_small.test.w -t sim 370ms±4.27 371ms±3.1 +1ms (+0.18%)⬜
jsii_small.test.w -t tf-aws 614ms±6.58 606ms±6.29 -8ms (-1.26%)⬜
empty.test.w -t sim 361ms±3.01 363ms±4.98 +2ms (+0.59%)⬜
empty.test.w -t tf-aws 589ms±4.7 601ms±3.36 +12ms (+1.99%)⬜
hello_world.test.w -t sim 401ms±11.23 401ms±5.04 +1ms (+0.15%)⬜
hello_world.test.w -t tf-aws 1518ms±6.02 1513ms±6.22 -5ms (-0.35%)⬜

⬜ Within 1.5 standard deviations
🟩 Faster, Above 1.5 standard deviations
🟥 Slower, Above 1.5 standard deviations

Benchmarks may vary outside of normal expectations, especially when running in GitHub Actions CI.

Results
name mean min max moe sd
version 59ms 56ms 73ms 4ms 5ms
functions_1.test.w -t sim 401ms 383ms 464ms 16ms 23ms
functions_1.test.w -t tf-aws 828ms 814ms 843ms 8ms 11ms
jsii_big.test.w -t sim 2800ms 2767ms 2843ms 19ms 27ms
jsii_big.test.w -t tf-aws 3001ms 2981ms 3029ms 13ms 18ms
functions_10.test.w -t sim 494ms 463ms 510ms 10ms 13ms
functions_10.test.w -t tf-aws 2014ms 1975ms 2040ms 15ms 20ms
jsii_small.test.w -t sim 371ms 365ms 378ms 3ms 4ms
jsii_small.test.w -t tf-aws 606ms 595ms 624ms 6ms 9ms
empty.test.w -t sim 363ms 352ms 374ms 5ms 7ms
empty.test.w -t tf-aws 601ms 592ms 606ms 3ms 5ms
hello_world.test.w -t sim 401ms 389ms 411ms 5ms 7ms
hello_world.test.w -t tf-aws 1513ms 1499ms 1526ms 6ms 9ms
Last Updated (UTC) 2024-05-15 07:38

@mergify Mergify
Copy link
Contributor

mergify bot commented May 15, 2024

Thanks for contributing, @tsuf239! This PR will now be added to the merge queue, or immediately merged if tsuf/run-tests-in-batches is up-to-date with main and the queue is empty.

mergify bot added a commit that referenced this pull request May 15, 2024
@mergify
Merge of #6381
2d48b23
@mergify mergify bot mentioned this pull request May 15, 2024
Closed
14 tasks
@mergify mergify bot merged commit 4db5b91 into main May 15, 2024
15 checks passed
@mergify mergify bot deleted the tsuf/run-tests-in-batches branch May 15, 2024 07:50
@monadabot
Copy link
Contributor

Congrats! 🚀 This was released in Wing 0.73.45.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eladb eladb eladb left review comments

@MarkMcCulloh MarkMcCulloh MarkMcCulloh approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

math - tf-aws test fails with no space left on device
4 participants
@tsuf239 @monadabot @eladb @MarkMcCulloh