Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sync processes compatible with autoscaling #637

Open
jkemp101 opened this issue Jan 26, 2024 · 0 comments
Open

Make sync processes compatible with autoscaling #637

jkemp101 opened this issue Jan 26, 2024 · 0 comments

Comments

@jkemp101
Copy link
Member

jkemp101 commented Jan 26, 2024

Modern cloud infrastructures can auto scale compute resources to reduce costs and reap other benefits.

Accounts are currently assigned to a specific host by setting its sync_host property to the hostname the process is running on. This makes it very difficult to change the underlying compute resource which is likely a Docker container in modern infrastructures. Ideally the sync process wouldn't have to know or care what host/container it was running on.

Two potential options:

  1. Leverage the existing .desired_sync_host (see this script) to reassign accounts after a scaling event.
  2. Remove the concept of .sync_host completely from accounts and have them dynamically assigned to running sync processes. Use something like a redis hashring to assign accounts in an orderly fashion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant