-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature / Suggestion #237
Comments
Really good idea, will test and probably release this week. |
Another possible approach is to apply a decorator to invoke and invoke_stream functions that enforces a rate limit. This approach is more proactive but might require changes elsewhere to enable configuring (setting) the rate limit to enforce. Great work on this repo. |
@jonny7737 decorators are a great idea (maybe using tenacity). as for a timeline, im tinkering with a new concept and after putting that out will work on the retry logic as that seems to be a p0 for a number of use-cases |
Hope I have contributed in some small way. Thanks for listening. Keep up the great work! |
In phi/llm/anthropic/claude.py, I think the invoke and invoke_stream methods would benifit from a try-except around the return statement to catch rate limit errors with a backoff mechanism. Something like this might work [WARNING: UNTESTED CODE]
The text was updated successfully, but these errors were encountered: