[Bug] IDispatcher cannot be resolved when connection to Kafka fails

[Thijmen]

Describe the bug

Sample project: https://github.com/Thijmen/Brighter/tree/idispatcher-exception-poc/samples/KafkaTaskQueue/KafkaOutbox

When using

services.AddHostedService<ServiceActivatorHostedService>();

And later connecting to Kafka (with an invalid hostname, for example), the Exception to the user is not very clear.

Unhandled exception: System.InvalidOperationException: Unable to resolve service for type 'Paramore.Brighter.ServiceActivator.IDispatcher' while attempting to activate 'Paramore.Brighter.ServiceActivator.Extensions.Hosting.ServiceActivatorHostedService'.
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateArgumentCallSites(Type implementationType, CallSiteChain callSiteChain, ParameterInfo[] parameters, Boolean throwIfCallSiteNotFound)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateConstructorCallSite(ResultCache lifetime, Type serviceType, Type implementationType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateExact(ServiceDescriptor descriptor, Type serviceType, CallSiteChain callSiteChain, Int32 slot)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.TryCreateEnumerable(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateCallSite(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.GetCallSite(Type serviceType, CallSiteChain callSiteChain)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at KafkaOutbox.Program.Main(String[] args) in /Users/thijmen.stavenuiter/Code/brighter-fork/samples/KafkaTaskQueue/KafkaOutbox/Program.cs:line 12

If you dont use the hosted service, you'll get a better exception:

 Paramore.Brighter.ChannelFailureException: Error finding topic com.kafka.mapping
       ---> Confluent.Kafka.KafkaException: Local: Broker transport failure
         at Confluent.Kafka.Impl.SafeKafkaHandle.GetMetadata(Boolean allTopics, SafeTopicHandle topic, Int32 millisecondsTimeout)
         at Confluent.Kafka.AdminClient.GetMetadata(String topic, TimeSpan timeout)

To Reproduce

Open Sample project: https://github.com/Thijmen/Brighter/tree/idispatcher-exception-poc/samples/KafkaTaskQueue/KafkaOutbox
And run it :)

Exceptions (if any)

Unhandled exception: System.InvalidOperationException: Unable to resolve service for type 'Paramore.Brighter.ServiceActivator.IDispatcher' while attempting to activate 'Paramore.Brighter.ServiceActivator.Extensions.Hosting.ServiceActivatorHostedService'.

[Thijmen]

I'm all for opening a PR, however I think this requires internal discussion on how to achieve this. Should Brighter even inject the dispatcher eventhough it's possibly not available?

[iancooper]

There is some useful thinking here about environment checks, particularly as cli check: https://jeremydmiller.com/2019/10/01/environment-checks-and-better-command-line-abilities-for-your-net-core-application/ and it came from this Twitter thread which has some interesting observations: https://twitter.com/RogerAlsing/status/1636273862718398464?s=20 which could relate to this problem - how do we do better diagnostics. So for example we could register "env" checks from our transports and inbox/outboxes then search for and run them all, if you request.