Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synchronize dependency versioning on automation servers #1389

Open
melange396 opened this issue Feb 14, 2024 · 1 comment
Open

Synchronize dependency versioning on automation servers #1389

melange396 opened this issue Feb 14, 2024 · 1 comment
Labels
acquisition changes acquisition logic code health readability, maintainability, best practices, etc dependencies Pull requests that update a dependency file devops building, running, deploying, environment stuff, handy utils, repository-related, engineer QoL, etc

Comments

@melange396
Copy link
Collaborator

(This issue is based on this comment: #1356 (comment))

New versions of our own code are kept up-to-date on the prod and staging "automation" servers by our old friend https://github.com/cmu-delphi/github-deploy-repo (馃), which does not appear to have any support for maintaining requirements/dependencies (much less their versions). As such, installation and versioning for dependencies on those machines seems to only be done purposefully and manually by someone with proper insight and permissions/credentials.

We should try to keep our production and development environments in sync so that we don't run into inconsistent and confusing situations caused by differing versions. We have talked about this a number of times before for various reasons, but i don't think we ever came to a consensus. Some possibilities for addressing it include:

  • Run our production automation jobs out of our prebuilt docker containers instead of on "bare hardware". They will then have the same versions of requirements/dependencies as are in dev. This might be "the right way to do things" but it sounds complicated and is probably a longer-term project. It is worth noting that our API/web servers currently run in containers.
  • Add a pip step like this to the end of any github-deploy-repo runs. This sounds easy(ish?) to do, though there could be unforeseen consequences. It is also a temporary band-aid, since g-d-r is already in the crosshairs in Move acquisition deployment off of github-deploy-repo聽#1042.
  • Maybe get ansible to do it?

Other ideas and suggestions are welcome!

@melange396 melange396 added dependencies Pull requests that update a dependency file acquisition changes acquisition logic code health readability, maintainability, best practices, etc devops building, running, deploying, environment stuff, handy utils, repository-related, engineer QoL, etc labels Feb 14, 2024
@melange396
Copy link
Collaborator Author

I am thinking that the pip step will probably need to be done within a venv so different installations (like covidcast-indicators vs delphi-epidata vs ???) on the same machine don't/can't cause version conflicts with each other.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
acquisition changes acquisition logic code health readability, maintainability, best practices, etc dependencies Pull requests that update a dependency file devops building, running, deploying, environment stuff, handy utils, repository-related, engineer QoL, etc
Projects
None yet
Development

No branches or pull requests

1 participant