Missing branch coverage for switch statements in Cobertura report

[ReneSchumacher]

I'm using Istanbul to generate a Cobertura coverage report for TypeScript code and then use ReportGenerator to generate a coverage report from the Cobertura file. I noticed that the values reported by ReportGenerator differ from the values in the Cobertura file's summary attributes. After some analysis it was clear that the Cobertura file does not properly log branches created by switch statements, even though Instanbul apparently knows about them.

Here's a quick example:

The code (TypeScript) looks like this:

import Result from '../../Result';
import PolicyBase from '../PolicyBase';
import IRule from '../rules/IRule';
import FixedCoverageThresholdRule from './rules/FixedCoverageThresholdRule';
import CompareToBaselineCoverageRule from './rules/CompareToBaselineCoverageRule';
import CoverageValueReader from './CoverageValueReader';
import CoverageValue from './CoverageValue';

export default class CodeCoveragePolicy extends PolicyBase {
    public async run(): Promise<Result> {
        this.result.logger.log(this.parameters.getLocalizedMessage('EvaluateCoverage'));

        let coverageReader = new CoverageValueReader(this.parameters, this.result);
        let coverageValue = await coverageReader.getCoverageValue(this.parameters.getTaskVariable('System.TeamProject'), Number(this.parameters.getTaskVariable('Build.BuildId')), true);

        // The CoverageValueReader might fail the policy if explicit filtering is enabled and no data is found
        if (this.result.failed) {
            return this.result;
        }
        
        let policyRule: IRule<CoverageValue>;
        switch (this.parameters.getTaskInput('coverageFailOption', true).toUpperCase()) {
            case 'FIXED':
                policyRule = new FixedCoverageThresholdRule(this.parameters, this.result, coverageValue);
                break;
            case 'BUILD':
                policyRule = new CompareToBaselineCoverageRule(this.parameters, this.result, coverageValue);
                break;
            default:
                this.result.error(this.parameters.getLocalizedMessage('InvalidParameterValue', this.parameters.getTaskInput('warningFailOption', true), 'Fail Build On (coverageFailOption)'));
                break;
        }
        if (policyRule && await policyRule.evaluate()) {
            this.result.success(this.parameters.getLocalizedMessage('CoveragePassed', coverageValue.percentage, coverageValue.elements, coverageValue.valueLabel));
        }

        return this.result;
    }
}

The report shows this:

<class name="CodeCoveragePolicy.ts" filename="src\policies\codeCoverage\CodeCoveragePolicy.ts" line-rate="0.9" branch-rate="0.8889">
  <methods>
    <method name="(anonymous_6)" hits="15" signature="()V">
      <lines>
        <line number="10" hits="15"/>
      </lines>
    </method>
  </methods>
  <lines>
    <line number="2" hits="1" branch="false"/>
    <line number="4" hits="1" branch="false"/>
    <line number="5" hits="1" branch="false"/>
    <line number="6" hits="1" branch="false"/>
    <line number="9" hits="1" branch="false"/>
    <line number="11" hits="15" branch="false"/>
    <line number="13" hits="15" branch="false"/>
    <line number="14" hits="15" branch="false"/>
    <line number="17" hits="15" branch="true" condition-coverage="100% (2/2)"/>
    <line number="18" hits="1" branch="false"/>
    <line number="22" hits="14" branch="false"/>
    <line number="24" hits="4" branch="false"/>
    <line number="25" hits="4" branch="false"/>
    <line number="27" hits="10" branch="false"/>
    <line number="28" hits="10" branch="false"/>
    <line number="30" hits="0" branch="false"/>
    <line number="31" hits="0" branch="false"/>
    <line number="33" hits="14" branch="true" condition-coverage="100% (4/4)"/>
    <line number="34" hits="7" branch="false"/>
    <line number="37" hits="14" branch="false"/>
  </lines>
</class>

ReportGenerator correctly counts the lines with condition-coverage attributes, which results in a branch coverage of 100% (6 out of 6 branches covered), but according to the report only 88.89% have been covered. Those 88.89% are 7 out of 9 branches and that is correct as in addition to the two if statements that account for the 6 branches there are three switch cases of which the default case is not covered.

I believe that line number 22 (the switch statement line) should have a conditions child element describing the cases. At least this is how Cobertura itself reports switch statements. The Cobertura reporter does not include any code to add condition elements in the report.

To make sure the coverage is correctly reported, the Cobertura report generator should properly report the branch information for switch statements.

[coreyfarrell]

istanbuljs maintainers do not have the knowledge needed to provide fixes for the Cobertura report. Any fix to this report will need to be provided by a member of the community posting a PR including one/more tests covering the fix.

[ReneSchumacher]

Hmm.. I'll have a look at the reporter and see if I can fix it.

[ReneSchumacher]

@coreyfarrell: I analyzed the issue and it is actually not related to the Cobertura reporter (which I might still update to create a full Cobertura report) but to source mapping. The CoverageMap is correct for the JavaScript code. However, during transformation to a MappedCoverage the line information for switch statements is wrongly reported. It always has the line information of the first case block instead of the switch statement itself. Therefore, the branch and statement maps don't match.

I'll try to figure out where this comes from. The source map itself seems to be correct as I can debug my code correctly. It seems to be an issue in istanbul-source-maps after all.

[coreyfarrell]

Anyone who wants to help you would need to see an example of this breakage, preferably a link to a repository with a minimum reproduction. In particular you have not said what version of istanbul modules you are using, recent versions (including the latest semver-major) have had fixes to source map handling.

[ReneSchumacher]

A repro can be found at https://github.com/ReneSchumacher/istanbulCoberturaIssue.

The build process with gulp is a bit complicated as I wanted to be as close as possible to my actual project which has a more complex folder structure.

[ReneSchumacher]

I created PR #518 to fix this.