-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Throttling (only run N functions over a specific time) #372
Comments
Yeah some form of global rate limiting is key for us too. We rely on this in Elixir's Oban when working with third party APIs. |
I had some discussion on rate limiting with the team here, if anyone's interested: https://discord.com/channels/842170679536517141/845000011040555018/1109915630917922958 |
Done! |
@tonyhb that's exciting! Where can we find more information/documentation about the new feature? |
Re-opening this and aiming to clarify a few things:
This issue has been reopened and renamed to specifically discuss throttling. We're aiming to build throttling directly into function config. RE. a custom error for retrying at specific times, you can implement your own backoff by throwing the RetryAt header, as parsed here: https://github.com/inngest/inngest/blob/f223b24454970bee45515ad7f85d7c8b1eaa67c8/pkg/execution/state/driver_response.go#L175-L177/. This will be exposed in V3 of the SDK, allowing you to "back off" and control retries however you'd like. |
+1 on this. Would be really helpful for handling external APIs with rate limits |
* Update cron use case base with new design, ctas * Delete draft page for now
While my functions are serverless I have some dependancies which are not, for example I have an on-premise whatsApp server which can only handle 25 requests per second.
I fire 400 events and they happily invoke serverless functions which invoke the whatsApp API as fast as they can, error and then retry. All my messages go out eventually but it's not pretty.
Concurrency limits will help but rate limiting is a slightly different issue, it's concurrency x function invocation time and function invocation time is variable.
If my functions take 100ms to execute and I have concurrency of 100 (Not sure what latency the system has but that's a lot of requests per second)
Workarounds would be to set a concurrency x function time to be conservatively lower than the rate limit or to introduce an artificially delay
setTimeout(() => sendMessage(), 1000)
or to delay the functions to spread them over a longer period.Ideally the functions would error signalling to the queue to slow down in the same way the
NonRetriableError
tells the queue to not retry.The text was updated successfully, but these errors were encountered: