The automatic rate-limiting feature can unknowingly cripple endpoints if there's a bug in the user's system

[lailabougria]

Describe the suggested improvement

Is your improvement related to a problem? Please describe.

The automatic rate-limiting feature, doesn't allow users to filter the errors for which to invoke/allow this behavior.

This means that if there's a bug in user code, it could cause a wide range of exceptions in their environment. This triggers the automatic rate-limiting feature, limiting the endpoint to minimum processing power. However, the endpoint can never exit the automatic rate-limiting since there's a bug in a much-used message type, meaning that other message types that could be successfully processed aren't due to the concurrency being down.

This could, in turn, cause other exceptions, for example, the queue filling up, causing errors upstream.

Describe the suggested solution

Describe alternatives you've considered

• Providing an alerting system (metric, custom check, etc), allowing notification of prolonged rate-limiting behavior
• Invoking a critical error on the endpoint after a threshold of time in rate-limiting mode
• Allowing the user to control the types of exceptions for which to enable the automatic rate-limiting feature

Additional Context

No response