AMSI logging implemented on Linux

[rhubarb-geek-nz]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues.
• Refer to the FAQ.
• Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

The AMSI logging is still being performed on Linux even though there are no AMSI modules installed.

As demonstrated in #21473 this can lead to excessive heap usage

Expected behavior

#!/usr/bin/env pwsh
$env:__PSDumpAMSILogContent='1'
$bytes = new-object byte[] -ArgumentList @(,200554320)
$random = new-object Random
$random.NextBytes($bytes)

Should not print anything

Actual behavior

#!/usr/bin/env pwsh
$env:__PSDumpAMSILogContent='1'
$bytes = new-object byte[] -ArgumentList @(,200554320)
$random = new-object Random
$random.NextBytes($bytes)

Prints the following log

=== Amsi notification report content ===
<System.Random>.NextBytes(<System.Byte[]>)
=== Amsi notification report success: False ===

Error details

No actual error other than incorrect behaviour

Environment data

Name                           Value
----                           -----
PSVersion                      7.4.2
PSEdition                      Core
GitCommitId                    7.4.2
OS                             Debian GNU/Linux 11 (bullseye)
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

[iSazonov]

AMSI is called in internal static bool ReportContent() and only on Windows.
What you see is a debug code. It's not worth the effort to mask it on Unix. (Maybe it can even be useful there for MSFT developers who approved the code.)

[rhubarb-geek-nz]

What you see is a debug code. It's not worth the effort to mask it on Unix.

What the system is doing for every .NET invocation on Linux and macOS is linearising all the string arguments then throwing the result away. For every .NET invocation.

Perhaps you don't think it is an issue when casually typing away at a terminal, but when you are using scripts to automate operations in a server environment it is a pointless overhead.

Case in point, calling the System.Convert.FromBase64String with a 200 megabyte string on an Raspberry Pi 4 took over five and a half seconds when using

$bytes = [System.Convert]::FromBase64String( $base64 )

but when using a cmdlet calling exactly the same .NET API

$bytes = $base64 | ConvertFrom-Base64String

it took just over a second.

So that was a 500% slowdown all caused by the serialisation which was thrown away.

[mklement0]

@iSazonov, I agree with @rhubarb-geek-nz that there's no good reason to saddle user on Unix-like platform not only with unnecessary runtime overhead, but - as noted - in the case of passing large strings also with significant memory overhead.

It would be simple to execute the code in question only if env var. DumpLogAMSIContent is set to 1 on Unix, which would preserve the ability to debug on demand, while not burdening regular method calls:

That is, the following code:

PowerShell/src/System.Management.Automation/engine/runtime/Binding/Binders.cs

Lines 6938 to 6950 in c19b651

	// Expression block runs two expressions in order:
	// - Log method invocation to AMSI Notifications (can throw PSSecurityException)
	// - Invoke method
	string targetName = methodInfo.ReflectedType?.FullName ?? string.Empty;
	expr = Expression.Block(
	Expression.Call(
	CachedReflectionInfo.MemberInvocationLoggingOps_LogMemberInvocation,
	Expression.Constant(targetName),
	Expression.Constant(name),
	Expression.NewArrayInit(
	typeof(object),
	args.Select(static e => e.Expression.Cast(typeof(object))))),
	expr);

could be guarded by a preceding #if UNIX directive that sets a Boolean to determine whether the code should be invoked, along the lines of:

#if UNIX
  bool callAMSI = Environment.GetEnvironmentVariable("__PSDumpAMSILogContent")?.Trim() == "1";
#else
  bool callAMSI = true;
#endif
if (callAMSI)
{
   // ... code above
}

Note: The above illustrates the concept, but the value of callAMSI can and should be determined only once per session, during session initialization, and then cached.

[iSazonov]

For performance there is another issue. You can ask MSFT team today on monthly community call if you want to get answer quickly about #21473 performance.

[mklement0]

@iSazonov:

• The memory usage problem and the - much worse - runtime penalty discussed in [System.Convert]::FromBase64String causes memory leak with large strings #21473 is in the context of Windows, where actual AMSI API calls are currently performed unconditionally.

  • As long as an AMSI call must be made for every .NET method call made from PowerShell code, these memory + performance issues are unavoidable, although the only really pathological cases are .NET method calls involving arguments that are very large strings.

  • Possibly providing an opt-out of these AMSI calls on Windows - something that obviously has security implications - is the subject of yet another issue, How can I disable PSAMSIMethodInvocationLogging #21491

• The issue at hand is purely about Unix-like platforms, where no security considerations apply, due to the absence of an API analogous to the Windows-only AMSI, so the issue at hand can and should be decided separately.

[mklement0]

@iSazonov, in light of the above:

• The Resolution-Answered label is inappropriate.

• While it is appropriate to make the security WG review this, let me summarize what I think the net effect of this proposal is:

  • It will result in no functional change, and therefore doesn't affect security.
  • It will generally improve performance of all .NET method calls on all Unix-like platforms.

[rhubarb-geek-nz]

Although this was marked as answered, it was neither a solution or an acknowledgment that there is any kind of problem.

[SteveL-MSFT]

WG reviewed this and agreed that the .NET method invocation logging for AMSI should only apply to Windows. The change should be separate and not dependent on the env var.