[💡 Feature]: Align parallel implemention with Cucumber API

[nextlevelbeard]

Is your feature request related to a problem?

Since we enabled the Cucumber API to run the tests in #11069 ( and #11067, #11065 , #11010) we are disabling the native parallel implementation for Cucumber, meaning that for each WDIO worker, we invoke runCucumber and create an entire Cucumber test run.

That introduces many redundancies when taking advantage of many built-in features of Cucumber.

• If enabled with option publish, each WDIO worker indirectly publishes a Cucumber.IO report, instead of having just one per WDIO test run
• Any formatters/reporters that users specify in the configs Cucumber options will generate a set of different outcomes and consolidating that data is either not feasable or not small of a task (i.e. 20 specs trigger 20 HTML reports)

List of native Cucumber formatters/reporters

We should take into account that Cucumber hooks can and often do reference the @wdio/globals, therefore on top of importing support code we must pass this context to the workers as well, otherwise we can run into reference errors such as

Error: No browser instance registered. Don't import @wdio/globals outside of the WDIO testrunner context. Or you have two two different "@wdio/globals" packages installed.
    at Object.get (file:///Users/user/repos/repo/node_modules/@wdio/globals/build/index.js:15:23)
    at Scenario.doStuff (file:///Users/user/repos/repo/e2e/steps/hooks.steps.ts:49:36)

We must also consider aligning the WDIO retry mechanism with the Cucumber retry mechanism.

Describe the solution you'd like.

It should just work, no redundancies should be in place.

When WDIO users specify X maxInstances, the Cucumber parallel optional should be enabled and a maximum of X Cucumber workers wrapped in WDIO workers should run seamlessly.

Describe alternatives you've considered.

No response

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[christian-bromann]

@nextlevelbeard can you speak a little bit about how the native parallel implementation for Cucumber differs to the WebdriverIO one? Does it support running Scenarios in parallel as well?

[jan-molak]

@nextlevelbeard please note that the Cucumber publishQuiet option was deprecated, and publish doesn't seem to be actively promoted by the team either, so its future is rather unclear.

The parallel execution mode is an interesting aspect, though. You might be interested in cucumber/cucumber-js#2156 when considering your design approach, as I highlighted there several limitations concerning parallel execution via Cucumber. There's also cucumber/cucumber-js#1711 where we talked about options for running Cucumber programmatically, as well as serenity-js/serenity-js#1308 that started some of those conversations.

I'm actually wondering if in the case of Cucumber integration, it could make sense to use the Cucumber runner with its parallel capabilities, retry mechanism, reporting mechanism and so on instead of the WDIO local runner. But, to come up with some nice way of plugging WDIO into Cucumber? Maybe a custom WebdriverIOWorld responsible for managing the browsers?

Cucumber also has very good support from all major IDEs for running individual scenarios and features, which is something that WDIO could benefit from.

Whatever you decide, I'd be happy to collaborate on that :)

[christian-bromann]

I feel inclined to close this issue since I don't see a way to support this since Cucumber spins up multiple processes to execute itself in parallel and that is just not feasible given that WebdriverIO has to orchestrate this. I am happy to discuss the feature of running multiple scenarios in within a single spec in parallel. This however has been discussed in length in #3962 and we came to the conclusion that a file based parallelism approach seems to be the best approach. That said, if there are strong opinions and interest in a higher parallelization degree, I am all open for new discussions.

[jan-molak]

@christian-bromann & @nextlevelbeard - see cucumber/cucumber-js#2357

[christian-bromann]

if Cucumber makes orchestrating this easier, I am happy to bring this back.

[tibioneh]

This would indeed be a very useful feature.

[christian-bromann]

@tibioneh can you comment on why? Note that you have the option to split your scenarios in multiple feature file. Furthermore a lot of time is spend starting and stopping the browser, so we you run many small scenarios in parallel you won't be gaining much time. Can you elaborate on that?