Better installation experience

[timbussmann]

Describe the suggested improvement

NServiceBus endpoints require infrastructure to be created to operate correctly. Most commonly, this is related to the transport (e.g. queues, subscriptions, timeout infra, etc.) and the storage (e.g. saga/outbox tables) but other infrastructure requirements might be needed.

When running in a least-privilege environment (most common in modern systems), the application running might/should not have the necessary permissions to create the required infrastructure. Instead, it is expected to create the infrastructure upfront, typically as part of the deployment pipeline. This is a task that requires some extra steps with NServiceBus. Common recommendations might be:

• Start the endpoint with privileged mode using Endpoint.Start or Endpoint.Create with installers enabled (EnableInstallers). Both these approaches have disadvantages: Start actually started message processing, while Create doesn't fully initialize the transport, which, depending on the transport, might cause some transport infrastructure to not be created.
• The new Installer APIs available in NServiceBus 8.1. This approach only works as a standalone task as the used endpoint configuration can't be used to start the endpoint afterward (see EndpointConfiguration can't be reused when using Installer API #6575). Similar to Endpoint.Start/Create, this API needs to be called by custom user code.
• Create the necessary databases and transport infrastructure manually with custom scripts. This is laborious and risky.

Describe the suggested solution

Installation should be executable from a command line for easy integration into a deployment pipeline. This could be something like a special installation flag (like in the old host), or a dedicated tool/script that can be run. When running this step, it should be easy to provide the connection strings and account information that are required for the infrastructure setup.

SQL persistence already solves this fairly nicely by generating deployment scripts at build time that can be integrated into deployment pipelines.

When considering special tooling, respecting the user's endpoint configuration becomes a challenge, since that configuration is typically embedded into the application due to the nature of the code-first APIs. In addition, these APIs also encourage to resolve configuration values at runtime which makes it more difficult for other tooling to understand the desired endpoint configuration. This is in contrast to many (docker) container deployment scenarios that are more focused on "static" configuration files. Many NServiceBus configuration APIs are difficult to map to a static configuration equivalent though (e.g. callback and factories).

Describe alternatives you've considered

This spike enhances the generic host to have a built-in installation flag that can be used during deployment, e.g. dotnet run my-endpoint --nservicebus install. This would enable a simple deployment script that completes after the endpoint has set up the necessary infrastructure. However, the spike is using brittle workarounds since the generic host is not built for such scenarios and a more specialized tool might be a simpler approach.

Additional Context

No response