TestableOrchardCoreApps.md

Making an Orchard Core app testable

Tips on making specific features testable are under the "Creating tests" page but here are some tips for the whole Orchard Core app you want to test. For an example of a UI-tested Orchard Core application see Lombiq's Open-Source Orchard Core Extensions.

Note that certain features of the Lombiq UI Testing Toolbox need to be enabled from test code in addition to making the app testable. Check out OrchardCoreUITestExecutorConfiguration for that part of the configuration; this page is only about changes necessary in the app.

• Create recipes with test content, and import them by starting with a UI testing-specific setup recipe.

  • While you can run tests from an existing database, using recipes to create a test environment (that almost entirely doubles as a development environment) is more reliable. Keep in mind, that the data you test shouldn't change randomly, you can't assert on data coming from the export of a production app which is updated all the time.
  • Such recipes needn't be added to the web app like normal recipes; you can just add setup recipes to the test project's Recipes folder with Build Action = Content and Copy to Output Directory = Copy if newer. They'll be picked up by the app under test.
  • Using Auto Setup works too, just check out the samples project.

• In your web project do the following, allowing configuration of the app when launched for testing with the following piece of code in the app's Program class:

  var configuration = builder.Configuration;
  
  builder.Services
      .AddSingleton(configuration)
      .AddOrchardCms();

  If your app uses ASP.NET Core minimal APIs, then you'll also need to add a Program class:

  [SuppressMessage(
      "Design",
      "CA1050: Declare types in namespaces",
      Justification = "As described here: https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests.")]
  public partial class Program
  {
      protected Program()
      {
          // Nothing to do here.
      }
  }

• If you make use of shortcuts then add the Lombiq.Tests.UI.Shortcuts project (either from NuGet or as a Git submodule) as a reference to the root app project. The module will be enabled in case of UI testing.

  Note that since authentication-related shortcuts need an implementation available for Microsoft.AspNetCore.Identity.IRoleStore<OrchardCore.Security.IRole> the OrchardCore.Roles feature will also be enabled. If you want to use a different implementation then enable the Shortcuts module from the test recipe, as well as the alternative Roles implementation, as below:

  "steps": [
      {
          "name": "feature",
          "enable": [
              "Lombiq.Tests.UI.Shortcuts"
          ]
      }
  ]

• Tests should be self-contained and they shouldn't rely on any external dependencies like APIs or CDNs. It should be possible to run the app completely offline.

  • For static resources always provide local copies and make the CDN optional. Also disable CDN usage in the setup recipe:

    "steps": [
        {
            "name": "settings",
            "UseCdn": false
        }
    ]

  • For external web APIs you can implement mock API services in features only enabled in tests. Those features you can again enable in a test recipe. An alternative is to use a tool that provides fake APIs like JSON Server or Fake JSON Server. You can run such tools from the command line in the test's code, e.g. with the excellent CliWrap that the UI Testing Toolbox uses too.

• By default, the culture settings used when setting up an Orchard site depend on the host machine's culture. You want to make these settings consistent across all environments though so e.g. datetime and number formatting will be consistent. You can do this by enabling OrchardCore.Localization and configuring the culture in site settings from the setup recipe:

  "steps": [
      {
          "name": "settings",
          // To make sure that e.g. numbers and dates are formatted the same way on all machines we have to specify the
          // culture too.
          "LocalizationSettings": {
              "DefaultCulture": "en-US",
              "SupportedCultures": [
                  "en-US"
              ]
          }
      }
  ]

  Furthermore, when running your app under Windows (even if just during development), add the following to your app's appsettings.json or appsettings.Development.json file:

  {
    "OrchardCore": {
      "OrchardCore_Localization_CultureOptions": {
        "IgnoreSystemSettings": true
      }
    }
  }

  This enforces consistent date formats regardless of the host machine's settings, see the corresponding issue.

• Some features send out e-mails. You can test them with the Lombiq UI Testing Toolbox's built-in feature to run an isolated local SMTP server with a web UI. The OrchardCore.Email feature will be automatically enabled, as well as the rest of the configuration applied.

• If you want the site to use Azure Blob Storage then you have to do the following:

  • The OrchardCore.Media.Azure feature will be automatically enabled, as well as the rest of the configuration applied.

  • It's recommended that you use the Azurite emulator as the storage for tests, not a real Azure Blob Storage resource. This is used by the UI Testing Toolbox by default. Be sure that it's running when the tests are executing. You'll need at least v3.25.0 of Azurite (you can check this with azurite -v, otherwise you'll get "The API version 2022-11-02 is not supported by Azurite." exceptions).

  • If you want to use Blob Storage during local development then you can also configure it in your app's appsettings.json or appsettings.Development.json file like below but this is not necessary for UI testing:

    {
      "OrchardCore": {
        "OrchardCore_Media_Azure": {
          "ConnectionString": "UseDevelopmentStorage=true",
          "ContainerName": "media",
          "CreateContainer": true
        }
      }
    }