Skip to content

Latest commit

 

History

History
50 lines (37 loc) · 6.83 KB

SecurityScanning.md

File metadata and controls

50 lines (37 loc) · 6.83 KB
Raw

Security scanning with ZAP

Overview

You can create detailed security scans of your app with Zed Attack Proxy (ZAP) right from the Lombiq UI Testing Toolbox, with nice reports. ZAP is the world's most widely used web app security scanner, and a fellow open-source project we can recommend. See a demo video of this feature here.

Sample ZAP security scan report

  • The most important default ZAP scans, Baseline, Full, GraphQL, and OpenAPI scans are included and readily usable. Note that these are modified to be more applicable to Orchard Core apps run on localhost during a UI testing scenario (notably, ajaxSpider is removed, since most Orchard Core apps don't need it but it takes a lot of time, and rules as well as selected technologies are adjusted). If you want to scan remote (and especially production) apps, then you'll need to create your own scans based on ZAP's default ones. These can then be run from inside UI tests too.
  • You can assert on scan results and thus fail the test if there are security warnings.
  • Since we use ZAP's Automation Framework for configuration, you have complete and detailed control over how the scans are configured, but you can also start with a simple configuration available in the .NET API.
  • SARIF reports are available to integrate with other InfoSec tools.

Working with ZAP in the Lombiq UI Testing Toolbox

  • We recommend you first check out the related samples in the Lombiq.Tests.UI.Samples project.
  • If you're new to ZAP, you can start learning by checking out ZAP's getting started guide, as well as the ZAP Chat Videos. The documentation on ZAP's Automation Framework and the ZAP Chat 06 Automation Introduction video (as well as the subsequent videos about it in the series) will help you understand what we use under the hood to instruct ZAP, and will allow you to use your completely custom Automation Framework plans too.
  • Be aware that ZAP scans run its own spider or with an internally managed browser instance, not in the browser launched by the test.
  • While ZAP is fully managed for you, Docker needs to be available and running to host the ZAP instance. On your development machine, you can install Docker Desktop.
  • The full scan of a website with even just 1-200 pages can take 5-10 minutes. So, be careful to fine-tune the ZAP configuration to make it suitable for your app.

Limitations

On Windows-based GitHub runners the security tests always fail with the following error:

The docker.exe pull softwaresecurityproject/zap-stable:2.14.0 --quiet command failed with the output below. no matching manifest for windows/amd64 10.0.20348 in the manifest list entries.

This is because the Docker installation is configured to use Windows images, while the ZAP docker image is only available for Linux. If you rely on our Lombiq GitHub Actions then you can configure it like this to disable a test, in this case SecurityScanningTests:

  build-and-test:
    name: Build and Test
    # See https://github.com/Lombiq/GitHub-Actions/blob/dev/.github/workflows/build-and-test-orchard-core.yml.
    uses: Lombiq/GitHub-Actions/.github/workflows/build-and-test-orchard-core.yml@dev
    with:
      machine-types: '["windows-latest"]'
      test-filter: "FullyQualifiedName!~SecurityScanningTests"

Troubleshooting

  • Most common alerts in Orchard Core can be resolved by using the extension method in Lombiq Helpful Libraries like this: orchardCoreBuilder.ConfigureSecurityDefaults(allowInlineStyle: true).
  • If you're unsure what happens in a scan, run the ZAP desktop app and load the Automation Framework plan's YAML file into it. If you use the default scans, then these will be available under the build output directory (like bin/Debug) under SecurityScanning/AutomationFrameworkPlans. Then, you can open and run them as demonstrated in this video.
  • If an alert is a false positive, follow the official docs. You can use the alertFilter job to ignore alerts in very specific conditions. You can also access this via the .NET configuration API via SecurityScanConfiguration.MarkScanRuleAsFalsePositiveForUrlWithRegex().
  • ZAP didn't find everything in your app? By default, ZAP has a crawl depth of 5 for its standard spider and 10 for its AJAX spider. Set maxDepth (and maxChildren) for spider and maxCrawlDepth for spiderAjax.
  • Do you sometimes get slightly different scan results? This is normal, and ZAP can be inconsistent/appear random within limits, see the official docs page.
  • Is the active scan too slow?
    • You can find out which rules take the most time by adding a script displaying each rules' runtime with YamlDocumentExtensions.AddDisplayActiveScanRuleRuntimesScriptAfterActiveScan().
    • The "Cross Site Scripting (DOM Based)" active scan rule, unlike other rules, launches browsers and thus will take 1-2 orders of magnitude more time than other scans, usually causing the bulk of an active scan's runtime. Also see the official docs. You can tune it so it completes faster but still produces acceptable results for your app. You can do this from the Automation Framework plan's YAML file (see the samples on how you can use a custom one), or with SecurityScanConfiguration.ConfigureXssActiveScanRule().
    • In CI workflows, you might want to restrict how many scans run in parallel, if you have more than one. You can use xUnit's [Collection] attributes to have e.g. only two collections for such tests, thus allowing only two parallel scans.