Parameter parsing/passing: an unquoted argument that looks like a named parameter/value pair separated with ":" (colon) is broken in two in positional binding

[mklement0]

^{Related: #6291 and #6360}

Note: Unlike the linked issues, the broken behavior described here only affects PowerShell commands, not also native executables.

Steps to reproduce

# Basic function without declared parameters that echoes its arguments via $Args
function Out-Args { $i = 0; $Args | % { 'arg[{0}]: {1}' -f $i++, $_ } }

# Advanced function that binds all arguments via `ValueFromRemainingArguments` and echoes them.
function Out-RemainingArgs { param([parameter(ValueFromRemainingArguments)] $otherArgs) $i = 0; $otherArgs | % { 'arg[{0}]: {1}' -f $i++, $_ } }


# Pass a -<param-nam>:<value> argument that looks like a named parameter, but isn't -
# it is bound anonymously, via $Arg / via `ValueFromRemainingArguments`
Out-Args -foo:bar
'---'
Out-RemainingArgs -foo:bar

Expected behavior

arg[0]: -foo:bar
---
arg[0]: -foo:bar

Actual behavior

arg[0]: -foo:
arg[1]: bar
---
arg[0]: -foo:
arg[1]: bar

That is, the single $Args / ValueFromRemainingArguments-bound argument was unexpectedly broken in two.

Note that, by contrast, the following cases DO work correctly:

• When passing tokens such as -foo:bar to external programs - because special effort is already being expended to make this case work (which is needed, because initially all such tokens are parsed as if they were named parameters, even if they later turn out not to be - see @BrucePay's comment below.)

  • However, when passing such arguments indirectly to external programs, via $Args or @Args, the breaking-in-two problem surfaces too - see Parameter parsing/passing: unquoted tokens that look like named arguments with colon as the separator are broken in two when passed indirectly via $Args / @Args #6360

• Had there been a named parameter $foo to bind to, things would have worked as expected (the token would have been recognized as a parameter-name/value pair: the -foo part would have been used to identify the target parameter, and the bar part would have become the parameter variable's value).

Environment data

PowerShell Core v6.0.1 on macOS 10.13.3
PowerShell Core v6.0.1 on Ubuntu 16.04.3 LTS
PowerShell Core v6.0.1 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
Windows PowerShell v5.1.15063.674 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)

[BrucePay]

Hi @mklement0 - this is actually by design. A ':' indicates the end of the parameter name and it doesn't need to be the last character in the string. If it's in the middle of the string, everything after the ':' is treated as the argument to that parameter. Handling parameters this way parallels how a lot of Windows command line tools parse their arguments.

[mklement0]

@BrucePay:

Understood, but that applies to named binding: if -Foo:bar binds to explicitly declared parameter
-Foo, the -Foo prefix is solely used for matching and eliminated during binding, so that the parameter variable $Foo then only receives bar.

By contrast, in anonymous binding I would expect a token such as -Foo:bar to be passed through as-is (after potential up-front expansions) - no assumptions should be made about what that token represents, even if it looks like a parameter-name/argument pair.

Note that the problem only arises with PowerShell functions, not with external programs: with the latter, commendably, -foo:bar is passed through as-is.

I don't know enough about PowerShell's parameter binding to assess whether the distinction I'm asking for presents a conceptual / technical challenge.

[lzybkr]

At the parser level, the parameter token ends with : and the value after that (space or not) is a new token. The : is just used by the parameter binder primarily when binding switch parameters where an argument is unexpected.

So from that point of view, this is easily explained and consistent.

There is of course the issue with native commands - there was some effort to not add a space between the arguments when that space did not appear in the original script.

There is a small problem with the current approach though - splatting to a native command will unfortunately add a space:

function e1 { echoargs.exe @args }
echoargs.exe -a:b
e1 -a:b

This outputs:

arg 0: <-a:b>

CommandLine: "EchoArgs.exe" -a:b

arg 0: <-a:>
arg 1: <b>

CommandLine: "EchoArgs.exe" -a: b

I suppose this is actually an independent bug, though changing behavior here would address the splatting issue as well (though it's not necessarily a good idea to change both - fixing splatting could be sufficient.)

[mklement0]

Thanks for the background information, @lzybkr.

So from that point of view, this is easily explained and consistent.
[...]
though it's not necessarily a good idea to change both

Understood, but my point is that, purely from a conceptual perspective, something that isn't a named argument - even though it may look like one - should be passed through as-is - irrespective of whether it is passed to an external program or to a PowerShell cmdlet/function/script.

The current behavior is inconsistent in that the token is parsed as a named argument - and altered in the process - yet ultimately passed as an unnamed one or, rather, two unnamed ones - and that's the problem.

Of course, a simple workaround is to quote the argument: Out-Argument '-foo:bar'

I've created a separate issue for the splatting bug: #6360

[BrucePay]

@mklement0 To be clear, parameters and arguments are only parsed once when the script is compiled. The command to run is not involved in parsing at all. Command resolution and parameter binding are separate steps done each time a command is executed, potentially long after the parse was completed. So, from the language perspective, -foo:bar is always parsed as parameter -foo and argument bar. For cmdlets (or script/functions with cmdlet binding) if the command doesn't define a foo parameter, specifying -foo is an error. For native commands, which have no parameter metadata, it's always treated as a value. There is currently a limited "hack" that allows the native command parameter binder to stitch literal arguments like -foo:bar back together by looking at the parameter token. See function BindParameters() in NativeCommandParameterBinder.cs

[mklement0]

@BrucePay: Thanks for this succinct peek behind the scenes and the pointer to the source code - much appreciated.

Based on #6360, as diagnosed by @lzybkr, we now know that the limited "hack" needs fixing for $Args-based parameter passing to external programs.

In the interest of a consistent user experience, my plea is to take this opportunity to apply the same "hack" to (directly or splatted) arguments passed to cmdlets/ functions too:

• calling non-advanced functions that have no parameters and use $Args
• binding to an advanced function's ValueFromRemainingArguments parameter

Something that - even though later, for technical reasons - turns out not to be a parameter, can be dealt with consistently only in one of two ways:

• EITHER reject the ultimate non-parameter as unbindable and report an error; tell users to use quoting in such cases.

• OR - and that's my plea - apply the limited "hack" consistently to piece the (potentially expanded) token halves back together, irrespective of whether the executable is an external one or a PowerShell cmdlet / function.

If you ultimately don't know what a token such as -foo:bar means, it makes no sense to pass it as two arguments - that is potentially more harmful than reporting an error.

I've updated the initial post to reflect the current state of our discussion.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.