5.3.0

[nohwnd]

Thanks @ArmaanMcleod, @fflaten and @johlju for all your contributions and support!

New Code Coverage mode

New way of measuring code coverage was added. It uses the core of Profiler to measure code coverage, instead of using breakpoints. This makes code coverage much much faster for bigger solutions. To enable it use UseBreakpoints option, and set it to $false:

$configuration.CodeCoverage.UseBreakpoints = $false

The option is marked as experimental, but from our tests it looks like the new approach is as accurate as the breakpoint based approach. The new approach does not yet work on PS3.

• Add Profiler based code coverage #1937

Configure output for failed tests

New option for controlling the verbosity of failed tests was added:

$configurationOutput.StackTraceVerbosity

With the folloing levels:

None - All Stack trace is hidden
FirstLine - First line of stack trace is shown
Filtered - Filtered lines that belong to Pester. This is default.
Full - All stack trace is shown

More info and more screenshots are in the PR.

• Limit output information for failed tests #1975

Shorter Detailed output

Output for discovery and filter was reduced in the Detailed and Diagnostic output, by moving discovery messages per file into the Diagnostic output, and hiding the Filter messages from the default Diagnostic output. All messages are still available via custom filter in $configuration.Debug.WriteDebugMessagesFrom.

More info and more screenshots are in the PR.

• Update output for Discovery and Filter #1969

Automatic CI detection

Pester automatically detects if it is running in AzureDevOps or GitHub Actions and augments the output to report errors correctly.

The default is to autodetect CI but it can be disabled by:

$configuration.Output.CIFormat = 'None'

More info and more screenshots are in the PR.

• Azure Devops & Github Actions error logging #1996

Override properties and methods via New-MockObject

New-MockObject can now override properties and methods on the created object. It counts the number of executions of a method and collects parameters provided to methods. It can also take an object as input and add properties and methods to it.

# Create an empty process object
$o = New-Object -TypeName 'System.Diagnostics.Process'
# Override Kill method to just return "killed"
$mockObject = New-MockObject -InputObject $o -Methods @{ Kill = { param($entireProcessTree) "killed" } }

$mockObject.Kill()
$mockObject.Kill($true)

# Inspect the history of invocations in `_<methodName>` 
$mockObject._kill

# Output:
# Call Arguments
# ---- ---------
#    1 {}
#    2 {True}

More examples are in the tests: https://github.com/pester/Pester/blob/main/tst/functions/New-MockObject.Tests.ps1
Real life-examples here: https://youtu.be/8GWqkGvV3H4?t=2517

• Add Ability to Override Properties on New-MockObject #1838

Skip remaining tests when any fails

A new option was added, that allows skipping the rest of the execution when any test fails. It can skip tests in the current block and all child blocks, as well as skip all remaining tests in the file, or in the whole run.

$configuration.Run.SkipRemainingOnFailure = 'Block'

The following scopes are supported:

Block - Skip all remaining tests in current bloc k(including child blocks and tests) after a failed test.
Container - Skip all remainng tests in the container (file or scriptblock) after a failed test.
Run - Skip all tests across all containers in a run after a failed test.
None - Default, keep original behaviour.

We used Skipped test result to skip those tests, but there was a lively discussion about how this should be achieved. Share your feedback if you have any.

More info and more screenshots are in the PR.

• Added Run.SkipRemainingOnFailure option #2023

Log conflicting varibles in Mocks

Mocks pick up all variables from your tests. When your variable names are the same as parameter names of a mocked command this can lead to unexpected results in ParameterFilter. We added a new message in the Diagnostic log that warns you that a variable is in conflict with a parameter name.

It 'filter should not pass due to test-variable' {
    $SamAccountName = 'abc'
    # Mock parameter filter below does not pass because SamAccountName is 'def'
    Get-Stuff -SamAccountName 'def'
    # Mock parameter filter below does "incorrectly" passes because SamAccountName is not specified here
    # but is set to 'abc' by the variable above
    Get-Stuff

    # This assertion fails because there was 1 call, and not 0 calls, because we had naming conflict betwween 
    # parameters and our variables.
    Should -Invoke Get-Stuff -Times 0 -Exactly -ParameterFilter { $SamAccountName -eq 'abc' }
}

• Log conflicting variables in Should -Invoke -ParameterFilter #1973

Better TestDrive performance

TestDrive internals were improved to save the location of test drive directory, instead of querying it every time. This greatly improved performance of any run.

When you don't need TestDrive or TestRegistry, you can disable both TestDrive and TestRegistry entirely, to get more performance gains:

$configuration.TestDrive.Enabled = $false
$configuration.TestRegistry.Enabled = $false

Disabling TestRegistry is also useful on restricted systems when access to Windows Registry is disabled by group policy.

• Use profiler to improve perf of test drive, name expanding and add-member #1955
• Add option to disable TestDrive and TestRegistry #2008

Additional fixes in documentation, output, and other bugs squashed:

• Fixed formatting for null elements in an array #1961
• Make InModuleScope-parameters available as variables in scriptblock #1957
• Improve operator-help returned from Get-ShouldOperator #1988
• Added Pester version message to detailed and diagnostic output #1962
• Removed StartMessage from output #1968
• Add formatdata to ensure list view for all sections in PesterConfiguration #1964
• Update reporttheme output to remove hard coded colour names #1934
• Set name in xml element testsuite #2034
• Added FileContentMatchMultilineExactly should assertion #2030
• Added Filter.ExcludeLine option #2016
• Add PSObject boxing support in PesterConfiguration #1978
• Add missing container-parameters to Data #1986
• Update InvokeVerifiable error message and add negate support #1976
• Adds support for Register-ArgumentCompleter #1979
• Fix throwing on missing folders #2012
• make Value-property setter private in options #1994
• Add analyzerrule for object-commands #1753
• Update New-PesterConfiguration help #1946
• Fix session state in Should -Invoke to use the correct session state #1954
• Update Pester.RSpec.ps1 #1951
• Save all files with UTF8BOM #1950
• Fix formatting in all files #1949
• Allow positional parameter in get-shouldoperator #1947
• Remove about links they break website MDX #1948
• Added missing format tests #1935

See full log here

---

This discussion was created from the release 5.3.0.

[fred-gagnon]

Indeed, the new Profiler is much faster than the breakpoint method. Our unit tests now run in about 10 minutes versus 17 minutes. However, I am seeing a few differences in coverage between the two methods. After processing the coverage file with Report Generator, I observe the following discrepency (on the left is the coverage using the Profiler while on the right is the coverage using Breakpoints):

What the profiler is reporting for these lines doesn't make much sense.

[nohwnd]

it's probably some special case I am not taking into account where the reported hit needs to be translated to a different number to agree with what the AST says is a command that should run. I will put this into tests and see what's wrong :)

Or you can try it: This is the source file that has edge cases I found so far for code coverage:

https://github.com/pester/Pester/blob/main/tst/CoverageTestFile.ps1

And these are tests that compare if the coverage is the same as before, there are also some negative cases where the previous coverage is less accurate, and those are commented out in the CoverageTestFile.ps1 source file:

https://github.com/pester/Pester/blob/main/tst/Pester.RSpec.Coverage.ts.ps1

[nohwnd]

If you'd decide to PR this, please target the rel/3.1.x branch 🙂