Add Exponential Retry Mechanism with Idempotency Headers

[Cliftonz]

In order to enhance the resilience and reliability of our SDK, we would like to introduce an Exponential Retry mechanism for retrying failed requests. Additionally, to ensure the idempotent processing of requests, it's vital to incorporate support for providing an Idempotency Key as per the draft specified in the HTTP Idempotency Key Header Field.

The key requirements for this implementation include:

1. Exponential Retry Mechanism:

   • The SDK should retry failed requests following an exponential backoff strategy to minimize the contention and impact on the systems involved.
   • The SDK should ensure that the retry mechanism is configurable (e.g., max retries, initial delay, maximum delay).

2. Idempotency Key Provisioning:

   • The SDK should allow for either automatic or manual provisioning of an Idempotency Key for each request.
   • The Idempotency Key should conform to either CUID, ULID, or UUID formats as specified in the draft.
   • The Idempotency Key should be included in the HTTP Header as Idempotency-Key and following the standards outlined in the draft.

3. Configuration and Documentation:

   • The SDK should provide configuration options for enabling/disabling the Exponential Retry mechanism and Idempotency Key provisioning.
   • Comprehensive documentation should be provided explaining the configuration options, operational behavior, and the benefits of using the Exponential Retry mechanism along with Idempotency Keys.

Acceptance Criteria:

• Implementation of the Exponential Retry mechanism with configurable parameters.
• Provisioning of Idempotency Keys, either automatically or manually, conforming to specified formats (CUID, ULID, or UUID).
• Adequate unit and integration testing to ensure the robustness and reliability of the implemented features.
• Comprehensive documentation on the usage and configuration of the Exponential Retry mechanism and Idempotency Key provisioning.
• Adherence to the specifications outlined in the HTTP Idempotency Key Header Field draft.

Update: You can reference the go-lang library to keep the method signature and configuration the same.
novuhq/go-novu#62

Please refer to the draft for further details on the HTTP Idempotency Key Header Field and ensure adherence to the specified standards while implementing this feature in the SDK.

[toddb]

If anyone looks at implementations, can I suggest that this is cross-cutting concern and should be injected. I can imagine a base implementation can be implemented as extensions.

• see https://github.com/App-vNext/Polly/wiki/Polly-and-HttpClientFactory
• see https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/implement-http-call-retries-exponential-backoff-polly

Note: there are two implementations: direct instantiation and injection. Only the later will work with this approach.

[Cliftonz]

@toddb Thanks for the input!