Setting up Madek with Ansible

Step by step procedure

Checkout / Upgrade the Master-Project

Checkout the Madek master-project https://github.com/Madek/madek including submodules (recursively). See either Initial Checkout or Upgrading below.

Descend into the deploy directory and continue with configuration of the inventory.

Initial Checkout

git clone --recursive https://github.com/Madek/madek.git

Upgrading

The version which is going to be deployed is exactly the one that is checked out in the master-project. So the master-project must be updated to upgrade you installation of Madek. From within the master-project:

git pull
git submodule update --init --recursive

It is possible to run into troubles if the submodule structure has changed. The following command might help. Backup your changes beforehand!

git reset --hard origin/master && git clean --force -fd
git submodule foreach --recursive 'git reset --hard HEAD && git clean --force -fd’

You can as well start from scratch.

Create an Inventory

See the ./inventories directory for examples.

Install or Upgrade Madek

bin/ansible-playbook -i $MY_INVENTORY_PATH  play_setup-and-deploy.yml

Create a Admin User and set the Password

bin/ansible-playbook -i my_hosts play_create-admin.yml -e 'admin_password=YOUR-SECRETE-PASSWORD'

This will create (or update) a user with the login admin and the given password.

Customization and Advanced Topics

The madek_master_secret

The setup generates a default madek_master_secret based on various properties of the target host and your application. These properties might be guessable or not, depending on the setup of your host. It is highly recommended to override the generated secret as shown in the invocations above.

Alternatives to the command line arguments are variables defined in the Ansible Inventory. This is convenient in particular in conjunction with the Ansible Vault.

Caveat: the Ansible rules of precedence should be understood when proceeding in this manner. At any rate, invoking

bin/ansible-playbook -i my_hosts play_info.yml --vault-password-file inventories/zhdk/vault-pass

(possibly with augmented arguments) will show the actual used values and is recommended to be consulted for confirmation.

Creating and Fetching Database Dumps

Example:

bin/ansible-playbook -i inventories/zhdk/staging-v3-pdata play_dump-and-fetch-data.yml

This will dump and fetch data as well as structure_and_data into tmp/{{ansible_host_name}}/tmp/ . Override dump_items to fetch only either of them.

From the datalayer directory (of the webapp) a restore would look like the following example:

rake db:pg:truncate_tables db:pg:data:restore FILE=../../../deploy/tmp/staging-v3-pdata/tmp/madek_data.pgbin

Configure Zencoder

See the examples.

Importing Data

Example

rsync -e ssh -Pav root@madek.*************.**:/var/local/madek-file-storage/* /opt/madekdata/

Secrets Encryption (ZHdK)

This is handled via https://github.com/elasticdog/transcrypt.

The transcrypt bash script is stored within this repo: ./bin/transcrypt.

To unlock the encrypted files run ./bin/unlock.

Unlocking is rarely necessary. Some cases include

• deployments to a ZHdK Server,
• changing ZHdK Secrets.

Run ./bin/transcrypt --flush-credentials to bring the files back to their encrypted state.

Key Rollover

See https://github.com/elasticdog/transcrypt?tab=readme-ov-file#rekeying for key rollover.

We store the symmetric key via gpg in the project. Run the following to encrypt a new key:

gpg -a --encrypt \
    -r Thomas.Schank@zhdk.ch \
    -r ci-exec-user@ci-executor-madek-2024 \
    -r jan.pistor@zhdk.ch \
    -r manuel.radl@zhdk.ch \
    -r matus.kmit@zhdk.ch \
    -r urs.vanbinsbergen@zhdk.ch \
    .transcrypt_key.txt