5.3.0

Thanks @ArmaanMcleod, @fflaten and @johlju for all your contributions and support!

New Code Coverage mode

New way of measuring code coverage was added. It uses the core of Profiler to measure code coverage, instead of using breakpoints. This makes code coverage much much faster for bigger solutions. To enable it use UseBreakpoints option, and set it to $false:

$configuration.CodeCoverage.UseBreakpoints = $false

The option is marked as experimental, but from our tests it looks like the new approach is as accurate as the breakpoint based approach. The new approach does not yet work on PS3.

• Add Profiler based code coverage #1937

Configure output for failed tests

New option for controlling the verbosity of failed tests was added:

$configurationOutput.StackTraceVerbosity

With the folloing levels:

None - All Stack trace is hidden
FirstLine - First line of stack trace is shown
Filtered - Filtered lines that belong to Pester. This is default.
Full - All stack trace is shown

More info and more screenshots are in the PR.

• Limit output information for failed tests #1975

Shorter Detailed output

Output for discovery and filter was reduced in the Detailed and Diagnostic output, by moving discovery messages per file into the Diagnostic output, and hiding the Filter messages from the default Diagnostic output. All messages are still available via custom filter in $configuration.Debug.WriteDebugMessagesFrom.

More info and more screenshots are in the PR.

• Update output for Discovery and Filter #1969

Automatic CI detection

Pester automatically detects if it is running in AzureDevOps or GitHub Actions and augments the output to report errors correctly.

The default is to autodetect CI but it can be disabled by:

$configuration.Output.CIFormat = 'None'

More info and more screenshots are in the PR.

• Azure Devops & Github Actions error logging #1996

Override properties and methods via New-MockObject

New-MockObject can now override properties and methods on the created object. It counts the number of executions of a method and collects parameters provided to methods. It can also take an object as input and add properties and methods to it.

# Create an empty process object
$o = New-Object -TypeName 'System.Diagnostics.Process'
# Override Kill method to just return "killed"
$mockObject = New-MockObject -InputObject $o -Methods @{ Kill = { param($entireProcessTree) "killed" } }

$mockObject.Kill()
$mockObject.Kill($true)

# Inspect the history of invocations in `_<methodName>` 
$mockObject._kill

# Output:
# Call Arguments
# ---- ---------
#    1 {}
#    2 {True}

More examples are in the tests: https://github.com/pester/Pester/blob/main/tst/functions/New-MockObject.Tests.ps1
Real life-examples here: https://youtu.be/8GWqkGvV3H4?t=2517

• Add Ability to Override Properties on New-MockObject #1838

Skip remaining tests when any fails

A new option was added, that allows skipping the rest of the execution when any test fails. It can skip tests in the current block and all child blocks, as well as skip all remaining tests in the file, or in the whole run.

$configuration.Run.SkipRemainingOnFailure = 'Block'

The following scopes are supported:

Block - Skip all remaining tests in current bloc k(including child blocks and tests) after a failed test.
Container - Skip all remainng tests in the container (file or scriptblock) after a failed test.
Run - Skip all tests across all containers in a run after a failed test.
None - Default, keep original behaviour.

We used Skipped test result to skip those tests, but there was a lively discussion about how this should be achieved. Share your feedback if you have any.

More info and more screenshots are in the PR.

• Added Run.SkipRemainingOnFailure option #2023

Log conflicting varibles in Mocks

Mocks pick up all variables from your tests. When your variable names are the same as parameter names of a mocked command this can lead to unexpected results in ParameterFilter. We added a new message in the Diagnostic log that warns you that a variable is in conflict with a parameter name.

It 'filter should not pass due to test-variable' {
    $SamAccountName = 'abc'
    # Mock parameter filter below does not pass because SamAccountName is 'def'
    Get-Stuff -SamAccountName 'def'
    # Mock parameter filter below does "incorrectly" passes because SamAccountName is not specified here
    # but is set to 'abc' by the variable above
    Get-Stuff

    # This assertion fails because there was 1 call, and not 0 calls, because we had naming conflict betwween 
    # parameters and our variables.
    Should -Invoke Get-Stuff -Times 0 -Exactly -ParameterFilter { $SamAccountName -eq 'abc' }
}

• Log conflicting variables in Should -Invoke -ParameterFilter #1973

Better TestDrive performance

TestDrive internals were improved to save the location of test drive directory, instead of querying it every time. This greatly improved performance of any run.

When you don't need TestDrive or TestRegistry, you can disable both TestDrive and TestRegistry entirely, to get more performance gains:

$configuration.TestDrive.Enabled = $false
$configuration.TestRegistry.Enabled = $false

Disabling TestRegistry is also useful on restricted systems when access to Windows Registry is disabled by group policy.

• Use profiler to improve perf of test drive, name expanding and add-member #1955
• Add option to disable TestDrive and TestRegistry #2008

Additional fixes in documentation, output, and other bugs squashed:

• Fixed formatting for null elements in an array #1961
• Make InModuleScope-parameters available as variables in scriptblock #1957
• Improve operator-help returned from Get-ShouldOperator #1988
• Added Pester version message to detailed and diagnostic output #1962
• Removed StartMessage from output #1968
• Add formatdata to ensure list view for all sections in PesterConfiguration #1964
• Update reporttheme output to remove hard coded colour names #1934
• Set name in xml element testsuite #2034
• Added FileContentMatchMultilineExactly should assertion #2030
• Added Filter.ExcludeLine option #2016
• Add PSObject boxing support in PesterConfiguration #1978
• Add missing container-parameters to Data #1986
• Update InvokeVerifiable error message and add negate support #1976
• Adds support for Register-ArgumentCompleter #1979
• Fix throwing on missing folders #2012
• make Value-property setter private in options #1994
• Add analyzerrule for object-commands #1753
• Update New-PesterConfiguration help #1946
• Fix session state in Should -Invoke to use the correct session state #1954
• Update Pester.RSpec.ps1 #1951
• Save all files with UTF8BOM #1950
• Fix formatting in all files #1949
• Allow positional parameter in get-shouldoperator #1947
• Remove about links they break website MDX #1948
• Added missing format tests #1935

See full log here

5.3.0-rc1

5.3.0-rc1

Issues Fixed

• Add more logging to BP CC and fix case sensitive new CC #2042

See full log here

(Full release notes will come in RC2)

5.3.0-beta1

5.3.0-beta1

Issues Fixed

• Add Ability to Override Properties on New-MockObject #1838
• Added Run.SkipRemainingOnFailure option #2023
• Added FileContentMatchMultilineExactly should assertion #2030
• Added Filter.ExcludeLine option #2016

• Fix broken link to pester.dev assertions page #2024
• Enable additional plugins to be added via $additionalPlugins internally #2020

See full log here

5.3.0-alpha5

5.3.0-alpha5

Issues Fixed

• Add PSObject boxing support in PesterConfiguration #1978
• Azure Devops & Github Actions error logging #1996
• Add missing container-parameters to Data #1986
• Update InvokeVerifiable error message and add negate support #1976
• Adds support for Register-ArgumentCompleter #1979
• Fix throwing on missing folders #2012
• Add option to disable TestDrive and TestRegistry #2008
• Fix failure on zero analyzed commands #2001
• Fix tracer state on cancelled Pester run with new CodeCoverage #1998
• Log conflicting variables in Should -Invoke -ParameterFilter #1973
• make Value-property setter private in options #1994
• Improve operator-help returned from Get-ShouldOperator #1988
• Limit output information for failed tests #1975
• Add analyzerrule for object-commands #1753
• Update reporttheme output to remove hard coded colour names #1934
• Update output for Discovery and Filter #1969
• Add formatdata to ensure list view for all sections in PesterConfiguration #1964
• Removed StartMessage from output #1968
• Build on PS5.1 #1966

See full log here

5.3.0-alpha4

• Fixes coverage of default values in param block when the new CC is used.

See in the previous releases how to test it and compare the old CC with the new, and let me know please.

5.3.0-alpha3

Broken build, don't use.

5.3.0-alpha2

• Improves the tracer based code coverage, please try it and let me know where it is different from the current CC. This is how you do it:

There is now $configuration.CodeCoverage.UseBreakpoints experimental flag that allows you to use Profiler based code coverage instead of breakpoints based code coverage.
Here is a fuller example of generating CC for your code base using both methods.

https://gist.github.com/nohwnd/3a29c8f4afc21329a146b5cba953d503

If you can please try it and compare the xmls. E.g. open both in vscode and use the Compare active file with... command from the command pallete. You should see only difference on the top in timestamps.

This should work from PS3 PS4 up. At least it does in our CI pipeline. (PS3 has some weird issues I can't figure out, like not being able to enumerate collections and so on, so use at your own risk.)

The tracer based CC also cannot process default values in parameters, so it reports them as missed.

5.3.0-alpha1...5.3.0-alpha2

4.10.2-beta1

• Added parameter for Alias in HaveParameter (#1463)
• Fixes #1028 - Code Coverage for proxy functions/steppablePipelines (#1494)
• Many gherkin runner fixes and improvements (#1276)

Full log 4.10.1...master

5.2.2

Some edge case fixes:

• Run Parameter filter in the correct scope Should -Invoke inside of InModuleScope instead running it where the mock was originally run (#1954)
• Update New-PesterConfiguration help (#1946)

Full log 5.2.1...5.2.2

5.3.0-alpha1

There is now $configuration.CodeCoverage.UseBreakpoints experimental flag that allows you to use Profiler based code coverage instead of breakpoints based code coverage.
Here is a fuller example of generating CC for your code base using both methods.

https://gist.github.com/nohwnd/3a29c8f4afc21329a146b5cba953d503

If you can please try it and compare the xmls. E.g. open both in vscode and use the Compare active file with... command from the command pallete. You should see only difference on the top in timestamps.

This should work from PS3 up. At least it does in our CI pipeline.