An incidentally [psobject]-wrapped strongly typed array isn't recognized as such in argument-based parameter binding

[mklement0]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues.
• Refer to the FAQ.
• Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

Note:

• This is yet another manifestation of Objects are situationally invisibly [psobject]-wrapped, sometimes causing unexpected behavior. #5579

# Create and import a sample cmdlet that requires [byte[]] input
Add-Type @'
using System.Management.Automation;
[Cmdlet("Get", "Foo")]
public class GetFooCommand : PSCmdlet
{
    [Parameter(Mandatory=true,ValueFromPipeline=true)]
    public byte[] InputObject;

    protected override void ProcessRecord()
    {
    }
}
'@ -PassThru | ForEach-Object Assembly | Import-Module

# Create two instances of a large byte array, one without and
# one with a [psobject] wrapper
$byteArrayUnwrapped = [byte[]]::new(64mb)
$byteArrayWrapped = New-Object byte[] 64mb  # equivalent, but with implicit [psobject] wrapper

'--- unwrapped'
Get-Foo -InputObject $byteArrayUnwrapped
'--- wrapped'
Get-Foo -InputObject $byteArrayWrapped # !! Excessively slow.
'--- done'

Note that parameter binding via the pipeline is not affected; that is,
, $byteArrayWrapped | Get-Foo works as intended (note the need to wrap $byteArrayWrapped in an aux., transitory array via the unary form of , so as to ensure that it is passed as a single object).

Expected behavior

Both calls should return virtually instantly, given that the argument type exactly matches the parameter type and that the cmdlet is otherwise a no-op.

Actual behavior

The second call, due to the invisible [psobject] wrapper resulting from use of New-Object to construct the array, takes an excessive amount of time to complete.

Error details

No response

Environment data

PowerShell 7.5.0-preview.2

Visuals

No response

[rhubarb-geek-nz]

The New-Object mechanism does not go through the AMSI logging

#!/usr/bin/env pwsh
$env:__PSDumpAMSILogContent='1'
$bytes1024 = new-object byte[] -ArgumentList @(,1024)
$bytes2048 = [byte[]]::new(2048)

Gives the output

=== Amsi notification report content ===
<System.Byte[]>.new(<2048>)
=== Amsi notification report success: False ===

So presume the New-Object mechanism is the preferred PowerShell mechanism.

[jborean93]

For creating an instance of a type using the ::new() syntax is preferred. Less overhead as it doesn’t need to deal with parameter binding and pipeline output.

[rhubarb-geek-nz]

For creating an instance of a type using the ::new() syntax is preferred. Less overhead as it doesn’t need to deal with parameter binding and pipeline output.

Then why does it go through the AMSI logging? I interpret anything that goes through AMSI logging as "this is untrusted code"

[jborean93]

AMSI is a security feature for tools to hook into the language and decide what can and can not run. PowerShell is one of the better languages for this as it has very powerful tools for both auditing and controlling what is run and while there might be issues with its implementation around performance it doesn't rule out the benefits it provides. Honestly with the recent talk about the base64 string conversion I can see some improvements around how it logs values but I think the fault lies more in how the data is provided to the API. Dealing with 200MB of raw bytes then compounding it as a base64 string is quite an expensive task to do which is what is exacerbating the issue even further. It should be a sign that you need to change how things are working to avoid providing such large data to an API (like streaming the data).

You'll find that PowerShell has a lot of logging mechanisms, not just for .NET method invocations but also for cmdlet invocations and the like You'll find some really nice blog posts talking about it in places like https://devblogs.microsoft.com/powershell/powershell-the-blue-team/. These features are critical for a language these days when it comes to security.

PowerShell Team

PowerShell ♥ the Blue Team - PowerShell Team

(Warning: Long blog post ahead! If you’d like to read (or share) this as a whitepaper, you can download it here: “Scripting Security and Protection Advances in Windows 10”). At Microsoft, we invest an enormous amount of time and energy managing world-class cloud services and incredibly large enterprise networks.

[rhubarb-geek-nz]

AMSI is a security feature for tools to hook into the language and decide what can and can not run.

An alternative opinion is that it is security theatre when security should be based on permissions and access control, not by a virus detection ambulance at the bottom of a cliff. The AMSI scanning as currently implemented is nonsense, there is no formal format for the messages to be interpreted by any true access control mechanism it is just noise and no context.

If it was truly an access control mechanism then both reflection invocations and all cmdlets would go through the mechanism.

What is worse is all private data in arguments is being passed on to 2nd and 3rd parties and I have absolutely no idea where that is going, what is being done with it or where it is logged.

If it is logged or passed on to any other process it is a violation of the protection and privacy of the process. boundary.

[mklement0]

@rhubarb-geek-nz, the AMSI angle is irrelevant to the issue at hand.

The issue at hand is simply about parameter binding.

• True, a New-Object-constructed instance isn't subject to AMSI logging, as explained in [System.Convert]::FromBase64String causes memory leak with large strings #21473 (comment).

• While a ::new() call - as any (implied) .NET method call - is subject to AMSI logging, this is unlikely to matter in terms of performance unless you happen to pass a large string as a constructor argument.

Whether AMSI logging should be performed at all on a per .NET-method call basis is an entirely separate matter, which you've already tried to address in:

• How can I disable PSAMSIMethodInvocationLogging #21491

[rhubarb-geek-nz]

My solution to the new object problem was an additional cast to remove the PSObject

$bytes = [byte[]](New-Object byte[] -ArgumentList @(,200554320))

[mklement0]

It won't gain you much beyond not having to repeat the type name, but the type-agnostic solution, using the intrinsic psobject property is:

$bytes = (New-Object byte[] 200554320).psobject.BaseObject