ContinuousDelivery
The Views3 team uses continuous integration / delivery (CI/CD) for its services and packages. The advantages of using CI/CD is that delivery is automated, which greaty reduces the workload and knowhow required to deploy. This, again, leads to a much higher throughput of code changes, making it possible to fix issues and include new features much quicker.
The main "trigger" in the views3 deployment change is Git(hub) commits. When a commit is made, often to the master branch or with a specific tag, the machinery springs into action and builds a Docker image or a Python package and publishes it in the right place.
The CI/CD platform used by the team is Github Actions. It is important that all core developers get a good understanding of what Github Actions is, and how it works. In addition, Github Secrets is widely used to allow the CI/CD worker to authenticate to relevant sites to gain push rights.
Generally, the workflow file describing CI/CD for a repository is located under
.github/workflows/cicd.yml.
Views 3 is a service oriented system, consisting of Docker containers. The images that these containers use to run are publically available and hosted on a Docker Hub operated by the team: Views 3. Each service is available under the following tag scheme:
views3/{service_name}:{service_version / latest}
Whenever a commit is pushed to the repository containing the code for a service with a new git tag denoting the version, a CI/CD routine is triggered, causing the new version of the image to be built.
A service user called viewser has been set up for the team on Docker Hub.
This user's credentials are added to Github repositories using Github Secrets,
and given relevant permissions on Docker Hub as part of the viewser team in the
views3 organization. The relevant Github secrets settings are DOCKER_USERNAME
and DOCKER_PASSWORD (see
this file
for an example).
It is important to remember to add the proper permissions to the viewser
group, to allow viewser to upload new versions of the docker image to Docker
hub, as permissions in the views3 organization are given on a per-repository
basis. To do this, the repository needs to exist first: Create it by building
and pushing the repository from the command line:
docker build -t views3/{name}:{version}
docker push views3/{name}:{version}
Substituting name and version with appropriate values. Then, log in as an
owner on the Docker hub and give the viewser team read/write permissions to the
repository. Now CI/CD should work via Github Actions!
Packages are versioned using either Poetry, or in the setup.py for
repositories depending on Cython builds. Python repository CI/CD is triggered
by pushes to the main / master branches.
Authentication is done using PyPi tokens. Each python package has a separate
token, which is generated and then added to Github secrets in the
PYPI_API_TOKEN variable (see
this file
for an example]).