Skip to content

Running

Jump to bottom
peder2911 edited this page Jan 21, 2022 · 12 revisions

Running Views 3

This section teaches you how to run Views 3 locally. The skills taught here are fundamental for running Views anywhere, as the system for managing images and running containers is exactly the same for development and deployment.

System requirements

Since Views 3 runs in docker, it has a very light footprint in terms of system dependencies. All you need is to have docker, and docker-compose installed on your system with the following versions:

  • docker => 19.03.8
  • docker-compose >= 1.28.6

If you're on OSX, you should use Docker Desktop, which should give you the CLI features you need.

With these two components installed, you are ready to run Views 3 locally.

Running Views 3

Views 3 is a system of services. Each service runs in a separate process, and communicates with other services over HTTP. Services are all Docker containers, and can easily be put up using Docker Compose.

Get the code

git clone https://github.com/prio-data/views3 $HOME/views3

This oneliner clones the Views 3 repo into your home (user) directory. Of course, feel free to clone it to whatever folder you like. Note that Views 3 is a metarepository.

Fixing folder permissions

Services do not run as root in their containers. Therefore, some folders need specific permission settings before running. To set up these permissions, run these lines from the $HOME/views3 folder:

for FOLDER in logs cache
do
   mkdir $FOLDER ; sudo chown -R 1005:1005 $FOLDER
done

Get the configuration

Views 3 needs to be paired with a set of configuration files. These files are located in the views3_config repository, which is private. If you don't have access, contact an administrator.

To add the configuration files, run these lines (assuming views3 was cloned to $HOME/views3):

mkdir -p $HOME/views3_config
cd $HOME/views3_config
git clone https://github.com/prio-data/views3_config .
./install.py ../views3

You should see a happy message. For more information about views 3 configuration, see the main article.

Get a database certificate

Views 3 mounts the certificates located at $HOME/.postgresql. These certificates must be granted by an administrator, and placed in this folder before you can run Views 3.

Run the services

If you have followed the previous steps, you have all the prerequisites for running Views 3! Now, navigate to the Views 3 repository folder (again, $HOME/views3 if you've followed the tutorial verbatim) before running the following commands.

First, you must retrieve the images for each service. You do this by simply running:

docker-compose pull

Once you have the images, you can simply run

docker-compose up -d

Congratulations! You now have Views 3 running locally on your computer. Verify by running either of these commands:

# Should show each service running as a container
docker ps

Pointing viewser to your local instance

Before completing this step, take note of the current value of the REMOTE_URL setting, as it will change. Do so with the following line:

viewser config get REMOTE_URL | awk '{print $2}'

To make viewser (and other dependent tools) communicate with your local Views 3 instance, you need to alter the REMOTE_URL configuration setting to contain the URL of your local instance. The storefront service which is what viewser expects to reach at the REMOTE_URL address. The default local URL of this service when running Views 3 is http://0.0.0.0:4000.

This means that you need to run this line in order to point your viewser to the right place:

viewser config set REMOTE_URL http://0.0.0.0:4000

Now commands run with viewser will be directed towards your local Views 3. Check if it is working correctly by running viewser queryset list to see a list of currently available querysets.

Developing services

Next, you'll be interested to know how to develop new versions of the constituent services. This is done in four steps:

  1. Make code changes in the relevant repository
  2. Build a development image
  3. Update the docker-compose file
  4. docker-compose up -d

For example, if you want to make a code change to the base-data-retriever service, you would follow these steps:

  1. clone https://www.github.com/prio-data/base_data_retriever
  2. Make your code change
  3. Build the image with docker build -t views3/base-data-retriever:dev
  4. Change the base data retriever image version in the docker-compose file to :dev instead of whatever version is currently referenced.
  5. docker-compose up -d

docker ps should now show your :dev version running as views3_base-data-retriever. If the container keeps restarting, your code change broke something. Check docker logs views3_base-data-retriever to fix the error.

When you have made a change that is ready to be incorporated in the service, push your changes in a feature branch and submit a pull request. Once the PR is approved, a commit can be made with a version tag, which will trigger a build.

As a closing note, remember not to push the above change to the docker-compose file to the views3 repository. To undo the change, do git checkout docker-compose.yml.

For more information about (best) practices when developing images, see this article.

Clone this wiki locally