Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give up on quota exhaustion #44

Open
garloff opened this issue Aug 29, 2021 · 1 comment
Open

Give up on quota exhaustion #44

garloff opened this issue Aug 29, 2021 · 1 comment

Comments

@garloff
Copy link
Contributor

garloff commented Aug 29, 2021

When running out of quota, we produce lots of alarms without additional insight. This can happen due to failed cleanups. (The script already tries hard, so the obvious way to address this already has been exercised.)

Suggestion:

  • api_monitor.sh: detect out of quota
  • on detection: remember and slow down (sleep 120)
  • on repeated detection: slow down and stop iterations
  • run_in_loop.sh: it does already try to do extra cleanups, which might help
  • add detection for run time < 1hr and sleep for the rest, to restart out alarming no more than once per hour
@tibeer
Copy link

tibeer commented Apr 4, 2022

@garloff no assignee(s) set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants