Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run Tests in grouped order #66

Open
biohazard999 opened this issue Aug 3, 2020 · 0 comments
Open

Run Tests in grouped order #66

biohazard999 opened this issue Aug 3, 2020 · 0 comments
Labels
enhancement New feature or request help wanted Extra attention is needed Usability UX enhancements or issues about usability
Projects

Comments

@biohazard999
Copy link
Contributor

biohazard999 commented Aug 3, 2020

Currently the tests are collected in a plain queue. That isn't optimal for several reasons:

  1. It limits what strategy we use for testing on a scope level (run tests linear, in parallel, random order)
  2. It's confusing for the end user by default, because with multiple nested groups we don't dive into subgroups first. For example:
Describe("Foo", () =>
{
	It("#1", () => true);

	Describe("Bar", () =>
	{
		It("#2", () => true);

		Describe("Baz", () =>
		{
			It("#3", () => true);
		});

		It("#4", () => true);
	});

	It("#5", () => true);

	Describe("Buz", () =>
	{
		It("#6", () => true);
	});

	It("#7", () => true);
});

Currently the execution order is:

☺ [00:00:00.0121]  Foo #1
☺ [00:00:00.0001]  Foo #5
☺ [00:00:00.0001]  Foo #7
☺ [00:00:00.0001]  Foo Bar #2
☺ [00:00:00.0001]  Foo Bar #4
☺ [00:00:00.0001]  Foo Buz #6
☺ [00:00:00.0001]  Foo Bar Baz #3

I'm not 100% sure, but I think the most intuitive way I can think of execution order would be in declared order:

☺ [00:00:00.0121]  Foo #1
☺ [00:00:00.0001]  Foo Bar #2
☺ [00:00:00.0001]  Foo Bar Baz #3
☺ [00:00:00.0001]  Foo Bar #4
☺ [00:00:00.0001]  Foo #5
☺ [00:00:00.0001]  Foo Buz #6
☺ [00:00:00.0001]  Foo #7

@hazard999 what's your thougths about that?

Because the nature how tests are collected I think we need to reorder them before execution.
I think this should be really easy with a OrderTestGroupMiddleware.
The difficult thing would be though, reconstructing the order only from the queue it self, cause there is no way to find out the executed order for nested groups...

@biohazard999 biohazard999 added the enhancement New feature or request label Aug 3, 2020
@biohazard999 biohazard999 added this to To Do in 0.2.0 via automation Aug 3, 2020
@biohazard999 biohazard999 added the Usability UX enhancements or issues about usability label Aug 3, 2020
@biohazard999 biohazard999 added the help wanted Extra attention is needed label Aug 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed Usability UX enhancements or issues about usability
Projects
0.2.0
  
To Do
Development

No branches or pull requests

1 participant