Add pwshw for console-less PowerShell on Windows

[SteveL-MSFT]

PR Summary

.NET Core 3.0 supports building WinExe type which doesn't bind to console host. This means that pwshw.exe launches like any GUI app (think notepad) and stdin/stdout/stderr isn't bound to pwshw.exe from wherever it is started. This is conceptually equivalent to the old wscript.exe and cscript.exe tools.

Build script updated to also build a new powershell-win-core-w project as we still need the console pwsh.exe built. The resulting pwshw.exe and other necessary files are copied to the pwsh.exe folder.

This is exposed as an experimental feature although it can't be disabled as it's a binary file. pwshw is never attached a console, so -OutputLog is added to the host to write output to a file making it easier to trouble issues. pwshw is always non-interactive even if the -interactive switch is used. ConsoleHost UserInterfaces updated to write to OutputLog if set. Code paths that try to access ActiveScreenBuffer have try...catch to fail silently when there is no console. Tests updated to account for being run within pwshw where necessary. Tests using pwsh and expecting console output is left as-is. CI is updated to run all CI and Other tests both elevated and unelevated under this new host on Windows.

NOTE: scripts that call console APIs will fail as those APIs will throw that there is no console

PR Context

Fix #3028

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
• Breaking changes
  • None
  • OR
  • Experimental feature(s) needed
    • Experimental feature name(s): PSConsoleLessHost
• User-facing changes
  • Not Applicable
  • OR
  • Documentation needed
    • Issue filed: Doc new pwshw.exe for PS7 MicrosoftDocs/PowerShell-Docs#5039
• Testing - New and feature
  • N/A or can only be tested interactively
  • OR
  • Make sure you've added a new test if existing tests do not effectively test the code changed
• Tooling
  • I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
  • OR
  • I have considered the user experience from a tooling perspective and enumerated concerns in the summary. This may include:
    • Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode (which runs in a different PS Host).
    • Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
    • Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
    • Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).

[RDIL]

🎉

[iSazonov]

@SteveL-MSFT Re PR description - The PR doesn't fix initial request in #3028, it is implement another case.
To address #3028 I created #10965. I think we could implement pwshw in #10965 in the same way as we did an alias for preview-pwsh.

[ghost]

@iSazonov The consensus in the discussion for #3028 is that the solution is to create a pwshw variant instead of continuing to use -WindowStyle Hidden, and this PR does that fairly cleanly.

[iSazonov]

@alexbuzzbee pwshw is only one option from many in the complex topic. My PR seems add more flexibility and allow to experiment with console, non-console, pseudo-console and GUI modes, and also fix -WindowStyle Hidden scenario.

[SteveL-MSFT]

@iSazonov I don't think the two are mutually exclusive

[iSazonov]

@SteveL-MSFT I updated a description in my PR to clarify my vision.

[adityapatwardhan]

@SteveL-MSFT Some packaging changes are needed:

1. Include pwshw for signing:

   PowerShell/tools/releaseBuild/signing.xml

   Line 25 in 417cbd7

   <file src="__INPATHROOT__\pwsh.exe" signType="AuthenticodeFormer" dest="__OUTPATHROOT__\pwsh.exe" />

2. WXS file needs to be updated
3. Maybe some changes are needed in https://github.com/PowerShell/PowerShell/blob/master/tools/packaging/packaging.psm1
4. Once these changes are done, please scheduled a full signed build on your branch in Azure DevOps.

GitHub

PowerShell/PowerShell

PowerShell for every system! Contribute to PowerShell/PowerShell development by creating an account on GitHub.

[daxian-dbw]

@SteveL-MSFT Could you please elaborate more about the expectation of pwshw? Like what will not work (will all console related operations not work?)

[SteveL-MSFT]

@daxian-dbw updated description

[iSazonov]

No tests running for pwshw.

[SteveL-MSFT]

@iSazonov what do you mean? The additional test was run: https://powershell.visualstudio.com/PowerShell/_build/results?buildId=37337&view=logs&jobId=65e467a7-433b-584d-c8f4-72fcd5732f9d&taskId=6a14d04a-9a6d-5c41-f839-531117eeb032&lineStart=857&lineEnd=858&colStart=1&colEnd=1

I don't think there's a need to rerun all tests with pwshw.exe, it's a compilation flag.

[iSazonov]

I don't think there's a need to rerun all tests with pwshw.exe, it's a compilation flag.

Why not if we expect that some features will do not work? Are you sure that all feature which must works really works? (In my PR (although another scenario) 66 tests failed in one job)

[SteveL-MSFT]

@iSazonov I'll see about running the tests manually, anything expecting a console won't work as that's the design of being consoleless. Otherwise, pwshw.exe doesn't do much by itself and still relies on ConsoleHost.dll.

[SteveL-MSFT]

@adityapatwardhan mscodehub build succeeded, will fix merge conflict after getting pwshw test results

[adityapatwardhan]

@SteveL-MSFT I tried the artifacts out of the mscodehub build. Does not seem to work :(

[SteveL-MSFT]

@adityapatwardhan pwshw.exe is a Windows App, so it's not associated with the console. You'll need to have it redirect to a file within your command: pwshw -nop -c '$PSVersionTable > ~/out.txt'

[SteveL-MSFT]

Ok, definitely a problem here. Write-Host immediately fails. Need to have it handle the case there is no host.

[iSazonov]

Ok, definitely a problem here. Write-Host immediately fails. Need to have it handle the case there is no host.

In my PR I follow these cases: (1) AllocConsole() for normal pwsh.exe scenario, (2) AttachConsole(-1) to attach to parent process console in Hidden scenario. This doesn't pass all tests on CIs, I still don't investigate what tests must pass but failed.

[SteveL-MSFT]

Tests completed in 3676.82s
Tests Passed: 7644,
Failed: 127,
Skipped: 335,
Pending: 106,
Inconclusive: 2

Had to use Invoke-Pester since Start-PSPester expects pwsh.exe. Will have to look through the failures.

[iSazonov]

I have still concerns about skipped tests. I'd expect that we add new positive test for pwshw if we skip a test for pwsh. But I did not review this in depth.

[DHowett]

FYI - #3028 (comment) talks about the console team's work to address this particulat deficiency. It might take some time to come to all Windows versions, but pwshw is a permanent fixture you may need to support forever if you commit to it. 😄

[SteveL-MSFT]

Closing this for now and waiting for Windows solution.

[iSazonov]

@SteveL-MSFT Even if we get this feature in Windows API in future we could merge 90% of this PR today and save your efforts.