Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using Run.Exit and Run.PassThru causes unexpected output to stdout #2300

Open
3 tasks done
codaamok opened this issue Feb 11, 2023 · 1 comment · May be fixed by #2303
Open
3 tasks done

Using Run.Exit and Run.PassThru causes unexpected output to stdout #2300

codaamok opened this issue Feb 11, 2023 · 1 comment · May be fixed by #2303
Labels

Comments

@codaamok
Copy link

Checklist

What is the issue?

While using the Run.PassThru and Run.Exit configuration options, a Pester object is unexpectedly dropped to the output stream if an exception is raised. The object is not in the output stream if an exception is not raised.

See screenshot below:

image

  • test.ps1 contains some blocks with a single test. throw is used in BeforeAll but it can be used in any block to reproduce the issue
  • The test script file is called from testrun.ps1 and the output should be assigned to $r within the scope of the script file
  • $r isn't dropped to the output stream, however we can see a Pester output object in the console which called testun.ps1

In order to reproduce, both PassThru and Exit must be true.

test.ps1

Describe 'Some test' {

    BeforeAll {
        Write-Host 'hello world from beforeall'
        throw 'some error'
    }

    AfterAll { 
        Write-Host 'hello world from afterall'
    }

    it 'should be true' {
        throw 'some error'
        Write-Host 'within a test'
        $true | Should -Be $true
    }
}

testrun.ps1

$Container = New-PesterContainer -Path 'C:\path\to\test.ps1'

$Configuration = New-PesterConfiguration -Hashtable @{
    Run = @{
        PassThru = $true
        Exit = $true
        Container = $Container
        SkipRemainingOnFailure = 'Container'
    }
    Output = @{
        Verbosity = 'None'
    }
    TestResult = @{
        Enabled = $false
        TestSuitName = 'abc'
    }
}

$r = Invoke-Pester -Configuration $Configuration

Expected Behavior

I expected to not see any output in the given code example.

Steps To Reproduce

  1. Create test.ps1 with the above code
  2. Create testrun.ps1 with the above code and update line 1 for path to test.ps1
  3. Call testrun.ps1

Describe your environment

Pester version     : 5.3.1 C:\Users\adamp\OneDrive - Adam Cook\Documents\PowerShell\Modules\Pester\5.3.1\Pester.psm1
PowerShell version : 7.3.2
OS version         : Microsoft Windows NT 10.0.22621.0

Possible Solution?

@fflaten suggests is discord to:

We should likely ignore passthru when run.exit is set and there is a failed test. It's returned as text so it wouldn't provide anything more than the normal colored output

@fflaten fflaten linked a pull request Feb 11, 2023 that will close this issue
5 tasks
@fflaten fflaten added the Bug label Feb 11, 2023
@fflaten
Copy link
Collaborator

fflaten commented Feb 11, 2023

Fixing this is a minor breaking change.

If you run Invoke-Pester ... directly in the terminal, the result would be printed since the process is killed before the result is assigned the variable. However, when you use Invoke-Pester inside a script like testrun.ps1, then Pester will only exit the script and not kill the process. That's when it gets messy:

  • If you call testrun.ps1 which includes $r = Invoke-Pester to catch the result (like your sample), then a failed run would not be assigned and unexpectedly returned to the screen.
  • If you do $r = ./testrun.ps1 instead then you'd be able to catch both success and failure atm.

Then again, is the fact that it works just considered luck -> bug? Run.Exit says Exit with non-zero exit code when the test run fails.. Personally I wouldn't expect to be able to post-process anything after reading that, but.. some people might.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants