Experimental feature PSNativeCommandArgumentPassing breaks the stop-parsing symbol

[mklement0]

Related: #15239, #15250, #15276 - see overview in #15143.

Seemingly, with experimental feature PSNativeCommandArgumentPassing in effect (and preference variable $PSNativeCommandArgumentPassing set to 'Standard'), --%, the stop-parsing symbol is broken in that it is seemingly ignored: escaping of " chars. that follow is now unexpectedly being performed:

Steps to reproduce

On Windows (Unix is affected too, but --% doesn't make much sense there anyway):

# This worked up to preview 4.
'foo' | find --% /v "" | Should -Be 'foo'

Expected behavior

The test should succeed.

Actual behavior

The test fails, because the " chars. are \-escaped:

Access denied - \\
InvalidResult: Expected 'foo', but got $null.

In effect, command line find /v "\"\"" is passed behind the scenes - instead of find /v ""

Note that with without --% the command now works ('foo' | find /v "", which previously failed).

While that is great, --% must not be broken.

Environment data

PowerShell Core 7.2.0-preview.5

[daxian-dbw]

@JamesWTruher Can you please take a look and see if this is by design? If not, please replace the Needs-Triage label with Issue-Bug.

[rmccampbell]

With some experimentation it looks to me like what is happening is the --% is not really being ignored, but it's being considered as one string literal and then passed as the first argument (which is then escaped), the same as if you wrote find '/v ""'.

Edit: that wasn't quite correct, it's passing the string as the full command line, but after doing the escaping. So the --% is bypassing powershell parsing, but not bypassing the escaping.

[JamesWTruher]

the new behavior enables this scenario directly:
'foo' | find /v "" and the additional --% is not needed.
The addition of the --% creates the second argument as a literal "" which is why the misbehavior occurs due to some other code paths. Because the legacy behavior constructs a string to be handed off, the quotes are part of that string. The empty string is (and should be) present in the argument list (new behavior) without the quotes.

I have noticed an additional issue because the quoted strings need to passed somewhat untampered with (which is occurring outside the powershell argumentlist handling).

for example:

'foo' | find /v "bl at"
foo

works just fine, but

'foo' | find /v "blat"
FIND: Parameter format not correct

does not. In both cases arglist is constructed correctly, and then handed off to the .NET api

DEBUG: 2022-07-19 11:37:05.0177 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\windows\system32\find.exe]
DEBUG: 2022-07-19 11:37:05.0180 ParameterBinding Information: 0 :     BIND cmd line arg [/v] to position [0]
DEBUG: 2022-07-19 11:37:05.0184 ParameterBinding Information: 0 :     BIND cmd line arg [bl at] to position [1]

DEBUG: 2022-07-19 11:38:02.8644 ParameterBinding Information: 0 : BIND NAMED native application line args [C:\windows\system32\find.exe]
DEBUG: 2022-07-19 11:38:02.8647 ParameterBinding Information: 0 :     BIND cmd line arg [/v] to position [0]
DEBUG: 2022-07-19 11:38:02.8653 ParameterBinding Information: 0 :     BIND cmd line arg [blat] to position [1]

I'm suspicious that this may actually be happening in the .net layer as there is some dodgy quote escapement that happens there. This matches the above observations because there is a space in the string, it would need to be quoted so i suspect it is internally. The second case would not need to be quoted and then we see that find.exe gets heartburn.

I've not been able to reproduce this with an other .exe (although my search has not been exhaustive), so it may be best to put find.exe into the list of legacy behavior (along with cmd.exe). If you have other examples of Windows tools which display similar aberrant behavior we can add those as well.

I don't agree that the new behavior "breaks the stop-parsing symbol" at all, we're constructing the argument list appropriately which is why I'm marking this by-design.

[rmccampbell]

Ignoring the oddities of find.exe, I think the --% syntax is still broken. By my understanding it should bypass both powershell parsing and any escaping, and pass the rest of the command line directly to the app, the same as it did in Legacy mode. This isn't the case though:

> $PSNativeCommandArgumentPassing = "Legacy"
> cmd --% /c echo "hi"
"hi"
> $PSNativeCommandArgumentPassing = "Standard"
> cmd --% /c echo "hi"
"\"hi\""

The Standard mode should match the Legacy mode here IMO.
(I'm not sure what the behavior should be if there are arguments before the --%, preferably those arguments would be properly escaped but I think that's probably irrelevant in most cases)

[ghost]

This issue has been marked as by-design and has not had any activity for 1 day. It has been closed for housekeeping purposes.

[rmccampbell]

Is this actually by design, given my previous comment? It doesn't seem to be usefully designed if so. That means that probably many cases would have to use cmd /c to get exact control over the command line

[mklement0]

@rmccampbell , please see

• Stop-parsing token is broken #18664