Code Coverage

[kiraLinden]

Are there any plans to include code coverage? I found gotwarlost/istanbul#132 but there doesn't seem to be a nice way to do this. (I'm specifically interested in cucumber)

[christian-bromann]

First of all I don't think that code coverage should be part of your integration tests. You should write unit tests first and make sure that you have a decent code coverage there. Selenium/WebdriverIO should then cover the end to end integration of your stack.

However you can do the same as described in that issue thread using WebdriverIO. Instead of using the syntax of selenium-webdriver, just use WebdriverIOs commands:

    client.execute("return window.__coverage__;").then(function (obj) {
        var str = JSON.stringify(obj);
        var options = {
            port: 8888,
            host: "localhost",
            path: "/coverage/client",
            method: "POST",
            headers: {
                "Content-Type": "application/json",
            }
        };
        var req = http.request(options, function (res) {
            console.log("\nFinished sending coverage data.");
            done();
        });
        req.write(str);
        req.end();
    });

But again, you shouldn't include code coverage in your integration tests.

[nanomosfet]

@christian-bromann

i am wondering why you believe coverage reports shouldn't be part of functional tests

[christian-bromann]

Because with WebdriverIO you write e2e tests that should test business cases from end to end (as the name states). Therefor you are not testing code but a specific user flow. You try to cover all business aspects of your app. Even though you technically could capture the amount of code that was executed in the frontend it doesn't make sense because it isn't interesting at all for your business cases.

With unit tests you actually do test code and therefor code coverage is important to see how well you are doing with your testing. In this case it has a value.

Feel free to capture coverage report in functional test but you will end up trying to make this report look good by writing more e2e tests instead of unit tests or integration tests. It is all about what you are trying to test and which tools you are using for it.

[kailin0512]

Hello @christian-bromann Thank you for your explanation.

One more thing I want to check is that if the code coverage is not included in the e2e tests, how do we know when the test cases are enough? Like without this we just need to trust developer and QE that feature has been fully automated but in reality they might not have covered all combinations of use cases.

And probably to separate unit tests code coverage with IT test code coverage, and report separately would also be a good remedy of the last point you are suggesting?

I noticed that with the support of devtools in V7, it made code coverage more possible than ever, but the bundler like webpack actually block the process/ease of getting code coverage, any thought on that?

Thanks