Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Account for rate limiting in Azure fetchers #2075

Open
Tracked by #2054
orouz opened this issue Apr 1, 2024 · 0 comments
Open
Tracked by #2054

Account for rate limiting in Azure fetchers #2075

orouz opened this issue Apr 1, 2024 · 0 comments
Assignees
Labels
8.15 candidate bug Something isn't working CIS Azure Team:Cloud Security Cloud Security team related

Comments

@orouz
Copy link
Collaborator

orouz commented Apr 1, 2024

Motivation

we need to account for rate limiting in our Azure fetchers to avoid losing resources we want to evaluate.
our usage of azure go sdk can be divided to two:

  1. graph explorer
    • there is no easy way to figure out rate limiting as different queries may end up consuming different quotas. instead, we should verify our usage adheres to azure's throttling guidelines.
  2. all other clients
    • implement rate limiting when applicable

Definition of done

  • account for rate limiting when possible
  • add a retry (with backoff) mechanism for failed requests

Out of scope

  • synced cloudbeat instances consuming the same quota
@orouz orouz added CIS Azure Team:Cloud Security Cloud Security team related labels Apr 1, 2024
@tehilashn tehilashn added the bug Something isn't working label May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.15 candidate bug Something isn't working CIS Azure Team:Cloud Security Cloud Security team related
Projects
None yet
Development

No branches or pull requests

3 participants