Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Cobertura coverage format #2298

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 7 additions & 3 deletions src/Main.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -382,6 +382,7 @@ function Invoke-Pester {
Default value is: JaCoCo.
Currently supported formats are:
- JaCoCo - this XML file format is compatible with Azure Devops, VSTS/TFS
- Cobertura - this XML file format is compatible with Azure Devops, VSTS/TFS
joeskeen marked this conversation as resolved.
Show resolved Hide resolved

The ReportGenerator tool can be used to consolidate multiple reports and provide code coverage reporting.
https://github.com/danielpalme/ReportGenerator
Expand Down Expand Up @@ -1154,10 +1155,13 @@ function Invoke-Pester {
$configuration = $run.PluginConfiguration.Coverage

if ("JaCoCo" -eq $configuration.OutputFormat -or "CoverageGutters" -eq $configuration.OutputFormat) {
[xml] $jaCoCoReport = [xml] (Get-JaCoCoReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport -Format $configuration.OutputFormat)
[xml] $coverageXmlReport = [xml] (Get-JaCoCoReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport -Format $configuration.OutputFormat)
}
elseif ("Cobertura" -eq $configuration.OutputFormat) {
[xml] $coverageXmlReport = [xml] (Get-CoberturaReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport)
}
else {
throw "CodeCoverage.CoverageFormat must be 'JaCoCo' or 'CoverageGutters', but it was $($configuration.OutputFormat), please review your configuration."
throw "CodeCoverage.CoverageFormat must be 'JaCoCo', 'CoverageGutters', or 'Cobertura' but it was $($configuration.OutputFormat), please review your configuration."
}

$settings = [Xml.XmlWriterSettings] @{
Expand All @@ -1172,7 +1176,7 @@ function Invoke-Pester {
$stringWriter = [Pester.Factory]::CreateStringWriter()
$xmlWriter = [Xml.XmlWriter]::Create($stringWriter, $settings)

$jaCocoReport.WriteContentTo($xmlWriter)
$coverageXmlReport.WriteContentTo($xmlWriter)

$xmlWriter.Flush()
$stringWriter.Flush()
Expand Down
2 changes: 1 addition & 1 deletion src/Pester.RSpec.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ function New-PesterConfiguration {
Enabled: Enable CodeCoverage.
Default value: $false

OutputFormat: Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters
OutputFormat: Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters, Cobertura
Default value: 'JaCoCo'

OutputPath: Path relative to the current directory where code coverage report is saved.
Expand Down
2 changes: 1 addition & 1 deletion src/csharp/Pester/CodeCoverageConfiguration.cs
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ public static CodeCoverageConfiguration ShallowClone(CodeCoverageConfiguration c
public CodeCoverageConfiguration() : base("CodeCoverage configuration.")
{
Enabled = new BoolOption("Enable CodeCoverage.", false);
OutputFormat = new StringOption("Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters", "JaCoCo");
OutputFormat = new StringOption("Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters, Cobertura", "JaCoCo");
OutputPath = new StringOption("Path relative to the current directory where code coverage report is saved.", "coverage.xml");
OutputEncoding = new StringOption("Encoding of the output file.", "UTF8");
Path = new StringArrayOption("Directories or files to be used for code coverage, by default the Path(s) from general settings are used, unless overridden here.", new string[0]);
Expand Down
240 changes: 236 additions & 4 deletions src/functions/Coverage.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -1043,6 +1043,229 @@ function Get-JaCoCoReportXml {
return $xml
}

function Get-CoberturaReportXml {
param (
[parameter(Mandatory = $true)]
$CommandCoverage,
[parameter(Mandatory = $true)]
[object] $CoverageReport,
[parameter(Mandatory = $true)]
[long] $TotalMilliseconds
)

if ($null -eq $CoverageReport -or ($pester.Show -eq [Pester.OutputTypes]::None) -or $CoverageReport.NumberOfCommandsAnalyzed -eq 0) {
return [string]::Empty
}

$now = & $SafeCommands['Get-Date']
$nineteenSeventy = & $SafeCommands['Get-Date'] -Date "01/01/1970"
[long] $endTime = [math]::Floor((New-TimeSpan -start $nineteenSeventy -end $now).TotalMilliseconds)
[long] $startTime = [math]::Floor($endTime - $TotalMilliseconds)

$commonRoot = Get-CommonParentPath -Path $CoverageReport.AnalyzedFiles

$allLines = @()
foreach ($line in $CoverageReport.HitCommands) {
$allLines += $line
}
foreach ($line in $CoverageReport.MissedCommands) {
$allLines += $line
}
joeskeen marked this conversation as resolved.
Show resolved Hide resolved

$parentDirSelector = { & $SafeCommands["Split-Path"] $_.File -Parent }

$packageGroups = $CommandCoverage | & $SafeCommands["Group-Object"] -Property $parentDirSelector
$packages = foreach ($packageGroup in $packageGroups) {
$classGroups = $packageGroup.Group | & $SafeCommands["Group-Object"] -Property File
$classes = foreach ($classGroup in $classGroups) {
$methodGroups = $classGroup.Group | & $SafeCommands["Group-Object"] -Property Function
$methods = foreach ($methodGroup in $methodGroups) {
if (!$methodGroup.Name) {
continue
}

$methodLineFilter = { $_.File -eq $classGroup.Name -and $_.Function -eq $methodGroup.Name }

$coveredLines = $CoverageReport.HitCommands `
| & $SafeCommands["Where-Object"] $methodLineFilter `
| & $SafeCommands["Group-Object"] -Property Line `
| New-LineNode

$lines = $allLines `
| & $SafeCommands["Where-Object"] $methodLineFilter `

Check notice

Code scanning / PSScriptAnalyzer

The built-in *-Object-cmdlets are slow compared to alternatives in .NET. To fix a violation of this rule, consider using an alterantive like `foreach`-keyword etc.`.

The built-in *-Object-cmdlets are slow compared to alternatives in .NET. To fix a violation of this rule, consider using an alterantive like `foreach`-keyword etc.`.
| & $SafeCommands["Group-Object"] -Property Line `
| New-LineNode

$hits = [int](
$lines.attributes.hits | & $SafeCommands["Measure-Object"] -Sum
).Sum

$method = [ordered]@{
name = 'method'
attributes = [ordered]@{
name = $methodGroup.Name
hits = $hits
}
children = [ordered]@{
lines = $lines | & $SafeCommands["Sort-Object"] { [int]$_.attributes.number }
}
totalLines = $lines.Length
coveredLines = $coveredLines.Length
}

$method
}

$methodsTotalLines = ($methods.totalLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$methodsCoveredLines = ($methods.coveredLines | & $SafeCommands["Measure-Object"] -Sum).Sum

$classLineFilter = { $_.File -eq $classGroup.Name -and -not $_.Function }

$coveredLines = $CoverageReport.HitCommands `
| & $SafeCommands["Where-Object"] $classLineFilter `

Check notice

Code scanning / PSScriptAnalyzer

The built-in *-Object-cmdlets are slow compared to alternatives in .NET. To fix a violation of this rule, consider using an alterantive like `foreach`-keyword etc.`.

The built-in *-Object-cmdlets are slow compared to alternatives in .NET. To fix a violation of this rule, consider using an alterantive like `foreach`-keyword etc.`.
| & $SafeCommands["Group-Object"] -Property Line `
| New-LineNode

$lines = $allLines `
| & $SafeCommands["Where-Object"] $classLineFilter `
Fixed Show fixed Hide fixed
| & $SafeCommands["Group-Object"] -Property Line `
| New-LineNode

$totalLines = $lines.Length + $methodsTotalLines
$coveredLines = $coveredLines.Length + $methodsCoveredLines
$lineRate = Get-LineRate -CoveredLines $coveredLines -TotalLines $totalLines

$class = [ordered]@{
name = 'class'
attributes = [ordered]@{
name = (& $SafeCommands["Split-Path"] $classGroup.Name -Leaf)
filename = $classGroup.Name.Substring($commonRoot.Length)
'line-rate' = $lineRate
'branch-rate' = 1
}
children = [ordered]@{
methods = $methods
lines = $lines | & $SafeCommands["Sort-Object"] { [int]$_.attributes.number }
}
totalLines = $totalLines
coveredLines = $coveredLines
}

$class
}

$totalLines = ($classes.totalLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$coveredLines = ($classes.coveredLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$lineRate = Get-LineRate -CoveredLines $coveredLines -TotalLines $totalLines

$package = [ordered]@{
name = 'package'
attributes = [ordered]@{
name = $packageGroup.Name.Substring($commonRoot.Length)
'line-rate' = $lineRate
'branch-rate' = 0
}
children = [ordered]@{
classes = $classes
}
totalLines = $totalLines
coveredLines = $coveredLines
}

$package
}

$totalLines = ($packages.totalLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$coveredLines = ($packages.coveredLines | & $SafeCommands["Measure-Object"] -Sum).Sum
$lineRate = Get-LineRate -CoveredLines $coveredLines -TotalLines $totalLines

$coverage = [ordered]@{
name = 'coverage'
attributes = [ordered]@{
'lines-valid' = $totalLines
'lines-covered' = $coveredLines
'line-rate' = $lineRate
'branches-valid' = 0
'branches-covered' = 0
'branch-rate' = 1
timestamp = $startTime
}
children = [ordered]@{
sources = [ordered]@{
name = 'source'
value = $commonRoot
}
packages = $packages
}
}

$xmlDeclaration = '<?xml version="1.0" ?>'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include encoding="UTF-8" like JaCoCo? Though I just noticed both hardcode this, while we actually make this configurable in CodeCoverage.OutputEncoding 😬

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting... so what should we do? Hard-code or try to get it from the settings?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd hardcode for now to make sure it's compatible with Cobertura-tools. At least we'd be consistent and support unicode-characters properly.

Then we'll fix both later in a separate issue as it's more a xml-thing.

$docType = '<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">'
$coverageXml = ConvertTo-XElement -Node $coverage
$document = "$xmlDeclaration`n$docType`n$coverageXml"

$document
}

function New-LineNode {
param(
[parameter(Mandatory = $true, ValueFromPipeline = $true)] [object] $LineGroup
)

process {
[ordered]@{
name = 'line'
attributes = [ordered]@{
number = [int]$LineGroup.Name
hits = [int]($LineGroup.Group.HitCount | & $SafeCommands["Measure-Object"] -Sum).Sum
}
}
}
}

function Get-LineRate {
param(
[parameter(Mandatory = $true)] [int] $CoveredLines,
[parameter(Mandatory = $true)] [int] $TotalLines
)

[double]$denominator = if ($TotalLines) { $TotalLines } else { 1 }

joeskeen marked this conversation as resolved.
Show resolved Hide resolved
$CoveredLines / $denominator
}

function ConvertTo-XElement {
param(
[parameter(Mandatory = $true)] [object] $Node
)

$element = [System.Xml.Linq.XElement]"<$($Node.name)/>"
if ($node.attributes) {
$attributes = $node.attributes
foreach ($attribute in $attributes.GetEnumerator()) {
$element.SetAttributeValue($attribute.Name, $attribute.Value)
}
}
if ($node.children) {
$children = $node.children
foreach ($child in $children.GetEnumerator()) {
$childElement = [System.Xml.Linq.XElement]"<$($child.Name)/>"
if ($child.Value.value) {
$childElement.SetValue($child.Value.value)
}
else {
$child.Value | & $SafeCommands["ForEach-Object"] {
Fixed Show fixed Hide fixed
$childXml = ConvertTo-XElement $_
$childElement.Add($childXml)
}
}
$element.Add($childElement)
}
}

$element
}

function Add-XmlElement {
param (
[parameter(Mandatory = $true)] [System.Xml.XmlNode] $Parent,
Expand All @@ -1051,14 +1274,23 @@ function Add-XmlElement {
)
$element = $Parent.AppendChild($Parent.OwnerDocument.CreateElement($Name))
if ($Attributes) {
foreach ($key in $Attributes.Keys) {
$attribute = $element.Attributes.Append($Parent.OwnerDocument.CreateAttribute($key))
$attribute.Value = $Attributes.$key
}
Add-XmlAttribute -Element $element -Attributes $Attributes
}
return $element
}

function Add-XmlAttribute {
param(
[parameter(Mandatory = $true)] [System.Xml.XmlNode] $Element,
[parameter(Mandatory = $true)] [System.Collections.IDictionary] $Attributes
)

foreach ($key in $Attributes.Keys) {
$attribute = $Element.Attributes.Append($Element.OwnerDocument.CreateAttribute($key))
$attribute.Value = $Attributes.$key
}
}

function Add-JaCoCoCounter {
param (
[parameter(Mandatory = $true)] [ValidateSet('Instruction', 'Line', 'Method', 'Class')] [string] $Type,
Expand Down