Repository for my MSc Thesis A Web Application for Compartmental Network Models Illustrated Using COVID-19
Execute the following steps for the basic setup of the project.
# if you don't have a copy yet, clone the repo:
git clone https://github.com/leo-pfeiffer/epi-sim.git
# move into the project root
cd epi-sim
python -m venv venv
source venv/bin/activate
make setupFor the setup of only the lib, see here.
For the setup/build of only the app, see here
We provide a Makefile with the following commands:
==============================================================
Available Make commands ======================================
==============================================================
For Docker commands: default is ENV=prod; for dev set ENV=dev)
==============================================================
requirements_lib Install pip requirements for lib only
requirements_app Install pip requirements for app only
requirements Install all pip requirements
env Generate .env files
setup Full setup of the project
setup_lib Setup of lib only
setup_app Setup of the project
test Run all unit tests
test_app Run unit tests for app
test_lib Run unit tests for lib
create_app_data Create data for web app from simulation results
batch_upload_experiments Upload experiment results
run Run web app with Flask dev server (dev only)
build Build web app with docker-compose
up Start existing container
down Stop running container
logs Attach docker compose logs
bash Start bash inside container
restart Restart container
clear-data Clear downloaded app data
help Print available commands
To see this output, run make help.
The project contains three top-level folders:
Contains the sources for the EpiSim web application.
app/app: Dash web applicationapp/nginx: NGINX server for production
Contains the source code for the epidemic models, networks, as well as the experiments run with these models.
lib/experiments: Contains Jupyter notebooks and utils for running the simulations, running analyses, and creating output for the thesislib/model: Contains epidemic models and networks
Contains setup scripts and .env file templates used in the setup.
We developed the project using Python 3.9, and recommend using this version. However, the project is backward compatible for Python versions 3.6+.
Since the app and the lib can be used independently of each other,
the requirements are listed in two separate requirements.txt files:
lib/requirements.txt: Requirements required for modellingapp/app/requirements.txt: Requirements required for the web application
If you want to install all requirements (both app and model) in one go, run
# from project root
make requirementsfrom within your virtual environment.
To build the project using docker compose, you also need those installed on your machine (Docker, Docker compose).
Critical features of the project were tested using unit tests.
You can either run all tests :
# from project root
make test... or only those for lib:
# from project root
make test_lib... or only those for app:
# from project root
make test_appFor setting up the modelling environment, do:
# Assuming you're in the project root and don't have a virtualenv there yet.
# You don't need to do this if you've already done the steps in Quickstart
python -m venv venv
source venv/bin/activate
make setup_libThis will install all lib requirements in your virtual environment and
generate the required .env files.
The model package in lib contains the source code defining the epidemic models (/compartmental_model), the networks (/network), as well as related utility functions.
The experiments package contains several Jupyter notebooks which were used to run different workflows described in the thesis, both for creating the mobility networks as well as running the simulations.
0_extraction: Extract mobility data from raw files (either local or from remote host 'http://209.182.235.76/data/msc/')1_transformation: Transform extracted data from last step into correct format and save to data repo2_create_and_simulate: Create networks and run metric calculations; Create model configurations and run simulations; Run simulations for model validation3_network_analysis: Visualise the network metrics result and compare mobility network to Google COVID-19 Mobility report4_model_validation: Visualise the results of the model validation simulations5_extension_influenza: Model simulations with Influenza parameterisation9_thesis_graphics: Generation of other graphics in the thesis
The utils directory contains various utility functinos that are used accross the notebooks. Most are them are fairly trivial, however the data_repo_api module is quite important.
The DataRepoAPI class is a simple client for the GitHub API. We use it to backup results produced by the output as well as to store the final simulation results as input for the web app.
The default repo URLs are automatically included in the lib/.env file when you run make env.
Reading from the (public) repository is possible without authentication. However, any write operations require you to set your own repository in the 'lib/.env' file and add your own GitHub Personal Access Token.
The web application can be run independently of lib.
You can either build it using docker compose (recommended for production),
or the built-in Werkzeug server that comes with Flask (recommended for development).
The web application can be built using docker compose using Nginx as the HTTP server and uWSGI as the application server.
To build the application, run
# from project root
make buildThis also generates the required .env files (if they don't exist yet). By default, this builds the production environment (ENV=prod). If you want
to build the dev environment instead, call
# from project root
make build ENV=devThe application starts at whichever port is specified in app/.env as an environment
variable for PORT (default 80) and DEV_PORT (default 4401).
Alternatively, you can also run the web app using the built-in Werkzeug server of Flask, which is not suitable for production but much simpler for development and test purposes.
# Assuming you're in the project root and don't have a virtualenv there yet.
# You don't need to do this if you've already done the steps in Quickstart
python -m venv venv
source venv/bin/activate
make setup_app
# the following starts the server
make runThis also generates the required .env files (if they don't exist yet).
Since this is meant for development only, the DEV_PORT specified in ./app/.env
as an environment variable is used.
For easy access and updating of the simulation data underlying the application,
the data is stored in a public data git repository hosted on GitHub.
Since the repository is public and the web app only requires read access, there is no direct need to change the remote source. However, one could set a different repository as source in the app/.env file. We recommend against this, since a lot of the application assumes the exact directory structure that is found in the default data repository.