Web Test Runner terminates randomly without logs

[jonkoops]

We're porting the Leaflet testsuite from Karma to Web Test Runner, and noticed that the test run seems to terminate seemingly at random. This can be reproduced by running the Leaflet tests in watch mode.

[jonkoops]

This issue is now also occurring in regular test runs on our CI.

[mourner]

We've narrowed it down to this line which is apparently causing WTR to crash (while it worked fine with Karma):

Object.defineProperty(window, 'navigator', {value: ...});

[jonkoops]

Aside from the crash itself, I think the main problem here is the lack of logging. It's not really possible to understand where this crash was coming from at a glance.

[bashmish]

Can you try to configure concurrency: 1 (https://modern-web.dev/docs/test-runner/cli-and-configuration/)?

As for the logs: you might need to setup browserLogs and filterBrowserLogs, but I think defaults should be good already. Maybe the logs are not displayed because of the CI env? Locally I used Playwright a lot with WTR and saw error logs a lot of times, so not sure what the problem can be, I suggest to debug a little bit with different settings.

[jonkoops]

We've already tried lowering the concurrency to no avail. We have been able to narrow down the problem to the fact that Web Test Runner requires navigator.userAgent and navigator.serviceWorker to function.

We should not be overwriting these in our test suite, but I still think error reporting by Web Test Runner could be improved here.