Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design change - pull data with a background worker. API endpoints should use a pre-filled table or view #434

Open
ITJamie opened this issue May 27, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@ITJamie
Copy link
Contributor

ITJamie commented May 27, 2022

Describe the feat you'd like
API endpoints like minions can be quite slow to load if data in the db is large OR if some minions have not had a highstate run in a long time (thus having to reach far back into db history)
I suggest making a table or view that contains all the latest computed data per minion.

Right now the viewset for minion is actually running a salt command (test.ping). I would suggest this moves into another function that gets called every x minutes by a queue worker / background task.

Some rough data. our salt db contains about 5 months of data. it is over 20gb.

In general it would be better that any of the "get" requests use pre-populated db data where possible. salt-call commands should only be run if making a change or triggering a "refresh" of the data being viewed.

@ITJamie ITJamie added the enhancement New feature or request label May 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant