Support for running tests without a browser (OSOE-365)

[Piedone]

In certain cases, we need a running app but not a browser (or at least not a full one), like for testing web APIs. For such cases, we could start tests without a browser and then use something like RestEase for APIs or ScrapySharp as a barebone browser. The browser is launched when AtataFactory.CreateDriver() runs, this is the only thing necessary to switch off. However, a browser would still sometimes be needed for the initial setup (though we could add a feature to only open a browser when the setup is run but not for the test itself, or utilize Auto Setup).

Jira issue

[sarahelsaig]

ScrapySharp is built on HTML Agility Pack (see dependencies tab here), which is excellent. I have used HAP for years so you will hear no complaints about it from me. But OC already references AngleSharp which is its competitor, so aren't we supposed to use that?

However, a browser would still sometimes be needed for the initial setup

We could make CLI based setup an option?

[0liver]

Afair, we used ScrapySharp for scraping back in discoverize. Really liked it.

[Piedone]

CLI-based setup can work too. We allow something like that with Setup Extensions since you only have to access a RESTful API there (i.e. no browser needed). Or, we could do some magic in ConfigureUITesting() and run the setup, much like Auto Setup, from IConfiguration data (and that way, we can supply setup parameters in the CLI, meaning we can also open them up in OrchardCoreUITestExecutorConfiguration).

This might actually be preferable for setting up the site in every case, unless we specifically want to test the setup screen.

No opinion on HAP vs AngleSharp, though having a different dependency than OC wouldn't really matter.

[Piedone]

With BrowserUITestContextExtensions added by Dávid here we now have the basic facilities for this, except for not starting a browser. And CreateAndSwitchToTenantAsync() can be used to set up a tenant without a browser.

[sarahelsaig]

The security scanning that was implemented in #322 would greatly benefit from this. It's already using its own internal crawler for the scan so the browser just takes up RAM and does nothing after setup. So now even a very stripped-down version of this feature would be beneficial.

As for the setup, consider adding a configuration to make it automatic/browserless even if the rest of the test uses a browser. I mean if the site uses auto-setup or stock unaltered OC setup, then we are UI testing the same stock features again and again. (the same situation as login was before the SignInDirectlyAsync extensions)
We could enable browser setup just in one test (e.g. BasicOrchardFeaturesShouldWork). The rest should be set up in programmatically to save time and power.

[Piedone]

With ExecuteTestAfterSetup the setup is run only once for each set of tests (that use the same setup operations, which is commonly the whole test suite), and then cached, so we're not wasting time on the setup repeatedly (and that single setup serves as the test of the setup itself).

However, for security scanning tests yes, not running a browser at all would be beneficial.

[sarahelsaig]

I know. But spinning up a Chrome process for the setup and using Selenium to type in the values to each field one by one still takes some time. I understand that it's not a huge cost, but on large projects it still adds up to machine time that could've been spent more productively.

[Piedone]

Sure.

[Piedone]

Yevgeniy told that in AtataFactory we could use UseDriverInitializationStage(AtataContextDriverInitializationStage.None) to not start a WebDriver but still have an Atata context.