Running

Running ViEWS 3

Prerequisites:

• docker => 19.03.8
• docker-compose >= 1.28.6
• Azure SP credentials

ViEWS 3 is a system of services. Each service runs in a separate process, and communicates with other services over HTTP. The system runs in Docker, and can easily be put up using the provided Docker Compose file.

Although service based architectures have a lot of advantages, both in terms of performance, maintainability and concern isolation, they can also be a bit tricky to develop on. Not to worry! This step by step guide shows you how.

Security

ViEWS 3 uses Azure both for storing Docker Images, and for storing cached data. The Azure resources are not generally accessible, and require authentication to reach.

Running

Clone the prio-data/views_integration repository. Then, inside the repo folder, do

docker-compose pull

This fetches the latest version of the images for each of the services. Once you have these, and you've placed the SP file in the ./sp folder, you can run

docker-compose up

Congratulations! You now have ViEWS3 running locally on your computer.

Developing

The way to do work on a service might seem a bit contrived at first, but makes sense if you think about it. The "issue" is that each service does its work in concert with the others, and must therefore be connected to the network that the other services are running on. In addition, to avoid fiddling too much with configuration settings, your development instance of the service should be reachable at the URL the other services are expecting to find it.

There are many ways to solve this, but one I've found to work well is to

1. Kill the running service you want to work on
2. Clone the code for the service you want to work on
3. Build its image
4. Run it in a container that
   • Is connected to the same network as the rest of the services
   • Has the same name as the service would have in the docker-compose
   • Mounts the local code folder, overwriting the folder inside the container
   • Runs uvicorn with --reload, which detects code-changes

This setup means that you're running the service in the right context, while it is also editable.

Example

I want to do work on the queryset-manager service. To do this, I:

1. I launch the composition with docker-compose up -d.
2. Kill the queryset-manager service with docker rm -f views_integration_queryset-manager_1
3. Clone the code for the services. From the views_integration repo folder this can be done with git submodule update --init --recursive
4. Build its image by running docker build -t my_local_qsm .

Then you must create a variable containing the path to your authentication env file:

export AUTH_FILE=/your/path/here

Now you're ready to run. The command to run this container is a bit long, but I've commented to show what's going on:

docker run \
   --rm \ # Ephemeral container
   --name queryset-manager # This makes other services able to find it (same name) \
   --network views-integration # Attach it to the compose network \
   -v $(pwd)/queryset_manager:/queryset_manager # Mount your code directory \
   -p 6000:80 \ # Publish port 80 on port 6000
   --env-file $AUTH_FILE \
   --env-file ../services.env \ # Service-names and other settings 
   my_local_qsm \
   uvicorn --reload --port 80 --host 0.0.0.0 queryset_manager.app:app # Run with hot-reload

This container is now reachable for the other services, and editable. Try editing the code in the queryset_manager folder, the container logs should show uvicorn restarting whenever you make changes. Try adding the following to queryset_manager/app.py:

@app.get("foo")
def handle_foo():
   return "bar"

Now curl 0.0.0.0:6000/foo will return bar.

Once you've made some cool changes, you can commit them to Github from the service repo. Remember to use branches. If you need to add new dependencies, add them to requirements.txt and rebuild the image.

Publishing changes

The process of updating ViEWS3 on the server is as follows:

1. Changes are committed to github
2. A new docker image is built and pushed to our internal Docker registry
3. Images are pulled on our server (docker-compose pull)
4. The services are restarted (docker-compose restart)

Currently, the service principal we're using doesn't have push rights, so pushing a new image to prioreg cannot be done by everyone. If you need to get in updates without this privilege, you could log onto hermes, clone the integration repo with --recurse-submodules, and run docker-compose build. Then when you run docker-compose up, you'll be running with fresh images.