Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PowerShell "Automation" for larger infrastructure. #165

Open
AlexMilotin opened this issue Dec 20, 2021 · 6 comments
Open

PowerShell "Automation" for larger infrastructure. #165

AlexMilotin opened this issue Dec 20, 2021 · 6 comments
Labels
discussion question or suggestion

Comments

@AlexMilotin
Copy link

Since i've been dealing with a lot of infrastructure without a proper deployment tool and i had to find a way to run the tool on multiple servers at once. Some of people might already found another way, but i feel in debt to at least provide my 2 cent.
You can define the variable locations as you please, this is what i used so far.

  1. Copy files to multiple servers -> using PS workflow
Workflow CopyStuff {
    $Computers = gc "D:\script\AleXM-Tests\Log4shell\Test-Multi\servers2.txt"
    $Source = "D:\script\AleXM-Tests\Log4shell\Test-Multi\test\*"
    $Destination = "C$\temp\"

    foreach -parallel ($Computer in $Computers){
        Copy-Item -Path $source -Destination "\\$Computer\$Destination" -Recurse
    }
}
CopyStuff
  1. Execute the log4j2-scan.exe with --scan-log4j1 --scan-logback --scan-zip --silent switches
    Report will look like below and will be saved on the server you execute it from in the location defined in $CSVFile variable.

image

<#
CVE-2021-44228 vulnerability scanning. It also supports nested JAR file scanning.

Script designed to search for a file (or pattern) across all fixed drives while excluding everything else
Simple rules:
    - Create a text file with your list of servers (no headers and they should all be FQDN)
    - Make sure they are of the same domain or you're at least using an account that has delegated rights in a trusted domain relation
    - Script runs under user credentials so your user HAS to have rights to connect to the remote servers.
    - Script uses invoke-command which means you have to enable RemotePowershell.
    - The script must be used from a Terminal server or a device that can connect to all of the targeted computers.
    - Save the script and then run it from a powershell terminal
    - Tools used : log4j2-scan.exe (Credits to : https://github.com/logpresso/CVE-2021-44228-Scanner)
    - MultiThread body script : Credits to Matei Daniel
    - Adjusted for targeted support : Alex Milotin
    - 


Usage example:
    .\Log4jScann-MultiThread.ps1 -computerlistSource servers.txt

NOTES:
    - By default the script starts 100 threads (connects to 100 computers at once). This can be modified by the -MAXJOBS parameter but will be capped at 300.

#>
param
(
	[Parameter(Mandatory = $true)]
	[string]$computerlistSource = 'servers.txt',
    [int]$MAXJOBS = 100
    #[string]$SearchFile
)
write-host
if ($MAXJOBS -gt 300) {
    Write-Warning "Number of jobs too high, capping at 300!"
    $MAXJOB = 300
    }
    else {
        $MAXJOB = $MAXJOBS
        }


<#
if ($SearchFile.Length -lt 4) {
    write-warning "File name too short or not specified. You can use asterisk (*) like this: log4j*.jar"
    write-host
    exit
    }
    #>
write-host -NoNewline -ForegroundColor Cyan "ComputerListSource: "
write-host -ForegroundColor Magenta $computerlistSource
write-host -NoNewline -ForegroundColor Cyan "MAXJOBs: "
write-host -ForegroundColor Magenta $MAXJOb
write-host
write-host "Delaying the start for 5 seconds. Review parameters..."
Start-Sleep -Seconds 5


[string]$global:LogText = ""
[string]$nline = ("-" * 128)


Write-Host "-------------------------------------------"
Write-Host "Serverlistesource: $computerlistSource"
Write-Host "PSScriptRoot: $PSScriptRoot"
Write-Host "Architecture: $env:PROCESSOR_ARCHITECTURE"
Write-Host ""

[String[]]$ServerList
$cmiOpt = New-CimSessionOption -Protocol DCOM

$computerlistSource = $computerlistSource.Trim().ToUpper()
$gd = get-date -format yyyy_MM_dd-hh_mm

if ($computerlistSource -match ".txt")
{
	[string]$thisPath = Split-Path -parent $PSCommandPath
	$localList = "$PSScriptRoot\$computerlistSource"
	Write-Host ("Read Computerlist-file = > $localList");
	$ServerList = Get-Content -Path "$localList" -ErrorAction Stop
}
else
{
	if ($computerlistSource -eq 'COMPUTERLIST')
	{
		[string]$thisPath = Split-Path -parent $PSCommandPath
		$localList = "$PSScriptRoot\Computerlist.txt"
		Write-Host ("Read Computerlist.txt = > $localList");
		$ServerList = Get-Content -Path "$localList" -ErrorAction Stop
	}
}


$JobNames = "log4j2-scan"
write-host -NoNewline "Removing old jobs ..."
Get-Job "$JobNames*" | Stop-Job
Get-Job "$JobNames*" | Remove-Job
write-host "done."


$i = $null


[int]$counter = 0
[int]$entryCount = $ServerList.count
[string]$loc = (Get-Item (Get-Location)).FullName
Write-Host ("Server in Serverlist: $entryCount entrys")

Write-Host "#####################################"
Write-Host ""
Write-Host ""
Write-Host ""
Write-Host "running with user: $Env:USERNAME - $Env:USERPROFILE"
Write-Host "$loc"


if ($entryCount -gt 0)
{
	    foreach ($ServernameEnty in $ServerList)
	    {
            $i++
		    $perc = 100 * $i/($entrycount)
		    $perc = [math]::Round($perc, 2)
		    $counter += 1
            Write-Progress -id 1 -Activity "Creating job for $computerlistSource ..Percent: $perc" -Status $ServernameEnty -PercentComplete (100 * $i/($entrycount))
            [string]$jn = "$jobnames.$ServernameEnty"
            
               $device = $args[1]
               $so = New-PSSessionOption -SkipRevocationCheck
               $sess = New-PSSession -ComputerName $ServernameEnty
               $result = Invoke-Command -Session $sess -ScriptBlock {
               
                $localPath = "C:\Temp\log4j2-scan.exe"
               
                    IF(Test-Path $localPath) {
                   
                    $FQDN = $env:COMPUTERNAME + "." + $env:USERDNSDOMAIN
                    $ScannerVersion = (& C:\Temp\log4j2-scan.exe --help | Select-String "Scanner") -replace 'Logpresso CVE-2021-44228 Vulnerability Scanner ','v'
                    $drives = (Get-Volume | where { ($_.Driveletter -gt 0) -and ($_.DriveType -eq "Fixed") }).DriveLetter
                    $AllItems = @()
                    foreach ($drive in $drives) {
                        $MountPoint = $drive + ":\"
                        Write-Host "Processing $MountPoint"
                        $Items = & C:\Temp\log4j2-scan.exe --drives $drive --scan-log4j1 --scan-logback --scan-zip --silent | Select-String "Found","Scanned" | Select Line
                        $CollectItems = @()
                        foreach ($item in $items) {
                            
                            $item | add-member -type NoteProperty -Name FQDN -Value $FQDN
                            $item | add-member -type NoteProperty -Name Version -Value $ScannerVersion
                            $item | Add-member -type NoteProperty -Name Scanned -Value $True
                            $item | add-member -type NoteProperty -Name Drive -Value $MountPoint
                            $CollectItems += $item
                        }
                        $AllItems += $CollectItems

                    }
                    $AllItems }

                    else {
                    $FQDN = $env:COMPUTERNAME + "." + $env:USERDNSDOMAIN
                    $drives = (Get-Volume | where { ($_.Driveletter -gt 0) -and ($_.DriveType -eq "Fixed") }).DriveLetter
                    $AllItems = @()
                    foreach ($drive in $drives) {
                    $MountPoint = $drive + ":\"
                    Write-Host "Processing $MountPoint"
                    #$CollectItems = @()

                    #$item = "Skipped"
                    $drive | add-member -type NoteProperty -Name FQDN -Value $FQDN
                    $drive | add-member -type NoteProperty -Name Scanned -Value $False
                    $drive | add-member -type NoteProperty -Name Drive -Value $MountPoint
                    $drive | add-member -type NoteProperty -Name Line -Value "N/A"
                    $drive | add-member -type NoteProperty -Name Version -Value "N/A"
                    
                    $AllItems += $drive
                    
                    } $AllItems
                    }
                    
                  
                  
                  } -ArgumentList $args[0] -AsJob -JobName $jn
               $result

            $getRunningJobsCount = (Get-Job "$JobNames*" | ? { $_.State -eq "Running" }).count

            while ($getRunningJobsCount -ge $MAXJOB)
		    {
			    #Write-Progress -Id 2 -Activity "Reached maximum number of threads ($($MAXJOB))..." -Status "Wait till it gets reduced.." -PercentComplete (100 * $i/($entrycount))
                Write-Progress -Id 2 -Activity "Reached maximum number of threads ($($MAXJOB))..." -Status "Wait till it gets reduced.." -PercentComplete (100 * $getRunningJobsCount/($MAXJOB))
            
			    write-host -NoNewline "Active jobs: "
			    write-host -NoNewline -ForegroundColor Yellow $getRunningJobsCount
			    write-host -NoNewline "`tCompleted jobs: "
			    write-host  -ForegroundColor green $getCompletedjobsCount
			    
			    Start-Sleep 5
			    $getRunningJobsCount = (Get-Job "$JobNames*" | ? { $_.State -eq "Running" }).count
                $getCompletedjobsCount = (Get-Job "$JobNames*" | ? { $_.State -eq "Completed" }).count
                $CurrentRunningJobs = $getRunningJobsCount
        
            }

            
        }
}

While (Get-Job "$JobNames*" | ? { $_.State -eq "Running" })
	        {
		        
                
		        $CurrentRunningJobs = (Get-Job "$JobNames*" | ? { $_.State -eq "Running" }).count
                if ($CurrentRunningJobs -le 0) {$CurrentRunningJobs = 0} 
		        Write-Progress -id 3 -Activity "Jobs are running, please wait." -Status "$($CurrentRunningJobs) jobs running" -PercentComplete (100 * ($MAXJOB - $CurrentRunningJobs)/$MAXJOB)
		        $JobStatus
		        Start-Sleep -Seconds 5
	        }

$JobNames = "log4j2-scan"
$gd = get-date -format yyyy_MM_dd-hh_mm

$Result = @()
foreach ($Job in (Get-Job | ? { $_.Name -like "$JobNames*" }))
{
	$JobResult = $null
	$JobResult = Receive-Job $Job
	$Result += $JobResult
	Remove-Job $Job
    Remove-Variable JobResult -ErrorAction SilentlyContinue -WarningAction SilentlyContinue
}
$CSVFile = $JobNames + "_" + $gd + ".csv"
$Result | select FQDN,Scanned,Drive,Version,Line | Export-Csv -NoTypeInformation $CSVFile

I hope it helps someone

@xeraph xeraph added the discussion question or suggestion label Dec 20, 2021
@xeraph
Copy link
Contributor

xeraph commented Dec 20, 2021

Wow. Thank you for sharing!

@ajddba
Copy link

ajddba commented Dec 20, 2021

@AlexMilotin - I have spent all weekend doing something Similar - how are you finding it speed wise? - i have got it scanning about 200 boxes in 8 hours.

@xeraph xeraph pinned this issue Dec 20, 2021
@AlexMilotin
Copy link
Author

@AlexMilotin - I have spent all weekend doing something Similar - how are you finding it speed wise? - i have got it scanning about 200 boxes in 8 hours.

I had quite a struggle in running it remotely. The first version using Background jobs took about 8 to 12 hours (depending on the server, when executing it remotely).
This version i can happily say that that took 3 hours for my entire infrastructure of 460 servers.
The difference seems that was done by creating the jobs in the PSSession as Remote Jobs and then the network factor was not a problem anymore.
There are 3 ways you can do it.
Measure-Command helped me a lot in find the fastest one.

Give it a try, using -MAXJob parameter and set it to 200. You'll be amazed.

@ajddba
Copy link

ajddba commented Dec 20, 2021

Hi @AlexMilotin - Tried the script - it ran in powershell fine but in the CSV it says it hasnt scanned - have you used a specific version of logpresso? Also in the 1st script are you just copying the EXE or the ps1 aswell?

@AlexMilotin
Copy link
Author

AlexMilotin commented Dec 22, 2021

Hi @ajddba . Yes starting from 2.3.2 and above.
The log4j-scan.exe needs to be located on C:\temp on each server

If you want to that in a faster way you can use PS workflow to copy the file. That if you have SMB enabled
Place the tool and the vcruntime140.dll in case you're missing VC++ on the servers (since this is a prerequisite) in D:\log4scan\tool\ . The workflow below will copy the 2 files from there to C:\Temp on the servers.
Afterwards you can use the script to run it remotely.
Be aware that you need at least PS V3.0 on the servers for this to work.

Workflow CopyStuff {
    $Computers = gc "D:\log4scan\servers.txt"
    $Source = "D:\log4scan\tool\*"
    $Destination = "C$\temp\"

    foreach -parallel ($Computer in $Computers){
        Copy-Item -Path $source -Destination "\\$Computer\$Destination" -Recurse
    }
}
CopyStuff

@barrygarry
Copy link

Legend thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion question or suggestion
Projects
None yet
Development

No branches or pull requests

4 participants