Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrape GitHub data with fewer API requests #12

Open
choldgraf opened this issue Feb 14, 2023 · 0 comments
Open

Scrape GitHub data with fewer API requests #12

choldgraf opened this issue Feb 14, 2023 · 0 comments

Comments

@choldgraf
Copy link
Member

Currently we use up quite a lot of API requests every time that we scrape the GitHub data. This means that if we update the site too many times, we'll hit our limit and have to wait a day. We should find a way to scrape this data in a way that uses fewer API calls. For example, by storing the data in a persistent cloud bucket and simply updating that data rather than scraping from scratch each time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant