Testing Guide Proposal 3

Guide for adding tests to run on the container orchestrator

In this type of implementation for testing containers, we execute the test by running the container to be tested on the Kubernetes container orchestrator. Running on the container orchestrator means that we replicate as much as possible a production environment, while allowing at the same time for more CPU/IO intensive testing that what we could achieve on the Jenkins slaves. We propose to store the logic of the test (ie. the code to retrieve test data and execute the test) in a bash script inside the original container, avoiding a secondary container and associated git repo. The following figure illustrates the complete cycle.

Following steps assume some level of understanding of bash scripting, usage of git and Docker to build a container.

Add testing logic to original container

Here we will add the testing steps, including data download, to a bash script. We will then add this bash script with the test steps to the original container that we want to test. Please note that the mere addition of the test bash script doesn't imply its execution on the original container, hence none of the data or additionally required packages will be added to the container. Only the text file containing the script will be added, which is negible in terms of size.

Now we will add all the logic of the test to the original container, this will be shown through an example with container-mzml2isa:

• Go to the local version of the container git repository that you intend to test
  • Clone it if you don't have it locally, for this example, that would be git clone https://github.com/phnmnl/container-mzml2isa or through your favourite git client.
  • If you had it previously, make sure that you're in the correct branch and up-to-date, which for this example would be:

cd container-mzml2isa
git checkout develop
git pull

• Optional Git-flow (if you do these, then you need to stick to git flow (cheatsheet) for the rest of the tutorial, unless that you understand git enough to get out of it):
  • Init the repository for git flow if it hasn't been initialized
  • Start a new feature for adding the test logic.

git flow init
# and answer with the defaults
# then, start the new feature
git flow feature start addTest1

• Create a runTest1.sh file with your testing logic, for this example, I wrote:

#!/bin/bash

# Get any dependencies needed for the testing part
apt-get update && apt-get install -y --no-install-recommends wget

# Get the data needed for testing
wget --no-check-certificate https://github.com/ISA-tools/mzml2isa-galaxy/raw/master/galaxy/mzml2isa/test-data/metabolomics_study.zip

# Prepare the environment (create necessary folders, uncompress files if needed, etc.)
mkdir /tmp/study-dir

# Run the tool with the test data
wrapper.py -html_file /tmp/study-dir/index.html -inputzip metabolomics_study.zip -out_dir /tmp/study-dir -study_title "test-study"

# Check that files were created/correctness
if ! [ -e "/tmp/study-dir/index.html" ]; then
	echo "HTML file doesn't exist"
	exit 1
fi

if ! [ -e "/tmp/study-dir/test-study/i_Investigation.txt" ]; then
	echo "ISA investigation file not found"
	exit 1
fi

echo "All files created successfully"

• Add file to the git repo: git add runTest1.sh
• Add file to the Dockerfile in the path and make it executable. For this example, I added the following to the Dockerfile:

ADD runTest1.sh /usr/local/bin/runTest1.sh
RUN chmod +x /usr/local/bin/runTest1.sh

• Make sure that you can build the Dockerfile locally. For this example

docker build -t pcm32/mzml2isa .

• Check that your test script works in the container. For this example

docker run -it --entrypoint=runTest1.sh pcm32/mzml2isa

• Add changes done to the Dockerfile for the commit: git add Dockerfile
• Commit changes through the cli git commit -m "Adds testing logic to container" or other git client.
• Optional Git-flow: close feature git flow feature finish addTest1
• Push changes to the remote repository git push, or through your favourite git client.
• The image should be rebuilt in the PhenoMeNal Jenkins instance on its own, and pushed to the registry. Check that the job is successful in Jenkins.

Create testing job in Jenkins

Now that we have embedded the testing logic on the original container, we will add this as a job in Jenkins to be executed on the Kubernetes container orchestrator, everytime after the original container is built. As an explanation, what will happen now is that Jenkins, through its slaves, will run helm to deploy the test job based on environment variables that you will setup in the Jenkins job for testing your container.

The following steps are meant to work on the PhenoMeNal Jenkins instance, which has been configured for this.

Steps for Jenkins job

• Create a new item, with name test-container-<your-tool-name> (for this example test-conatiner-mzml2isa), copying existing item test-template-type3, as shown below (names on the screenshots below might be outdated). Press "Ok".
• On the following page, go to Build triggers check Build after other projects are built and add the name of the project that normally builds the container that you want to test.
• Go to Build -> Inject environment variables and set the following environment variables on the text field accordingly (values for this example shown):
  • POD_PREFIX=mzml2isa in your case it should match your tool name
  • CONTAINER=container-registry.phenomenal-h2020.eu/phnmnl/mzml2isa:latest in you case it should match your container.
  • TEST_CMD=runTest1.sh This one needs to match the test script that you added to the container and that you want to execute.
• Press "Apply" and then "Save"

Finally, if you either make a commit to your container, or just build it manually, this test should be triggered after it.

For reference, how does it work

Just for reference, as you don't need to replicate any of the following points, for this to work, we have previously set up on our Jenkins instance:

• Jenkins slave nodes with the ability to execute kubectl, to be able to trigger executions on Kubernetes clusters.
• Added the Kubernetes cluster config file containing all necessary keys/certificates as a secret file on Jenkins.
• Added the helm client to slaves, and initialised helm on slaves to talk to the cluster. Helm is a package manager for Kubernetes.
• Created a helm template repo on https://phnmnl.github.io/helm-charts (not human viewable).
• Created a pod template for our test jobs.
• Created a bash script to execute everything, which uses environment variables that you will set up in Jenkins. This is downloaded by Jenkins for you to the slaves.