Skip to content

ci-for-research/self-hosted-runners

Repository files navigation

Self-hosted runners

Five recommendations for fair software from fair-software.nl Badges
1. Code repository GitHub badge
2. License License badge
3. Community registry Ansible Galaxy badge Research Software Directory
4. Enable citation DOI
5. Checklist N/A
Other best practices
Markdown Link Checker Check Markdown links

How to set up GitHub Action runners on self-hosted infrastructure

This repository explains how to set up a server for running continuous integration tests on other hardware than what GitHub provides. This can be useful when the code you want to test has special requirements, for example if

  • it needs a GPU to run (CUDA installation with ansible instructions)
  • it needs multiple nodes
  • testing requires data that needs to stay on-premises for privacy reasons or legal reasons
  • testing requires data that is too big to move
  • testing requires specific software

This guide distinguishes between the client and the server; the client is your own machine; the server is whichever machine runs the tests. For either side, we'll explain what configuration needs to be done. For people who just want to try out the instructions but don't have access to remote hardware, we included a few alternatives for running the server locally as well, through the use of virtualization (with VirtualBox) and containerization (with Docker).

For the client, we included instructions for Linux Ubuntu, Mac, and Windows; the server-side instructions all assume Linux Ubuntu.

Status Client OS Server hardware Runner
✔️ Completed Linux Ubuntu local machine via Docker link
✔️ Completed Linux Ubuntu local machine via Singularity link
✔️ Completed Linux Ubuntu local machine via Vagrant link
✔️ Completed Linux Ubuntu local machine via VirtualBox link
✔️ Completed Linux Ubuntu remote machine at SURF HPC Cloud link
⏳ WIP Mac local machine via Docker -
⏳ WIP Mac local machine via Vagrant -
⏳ WIP Mac local machine via VirtualBox -
⏳ WIP Mac remote machine at SURF HPC Cloud -
⏳ WIP Windows local machine via Docker -
✔️ Completed Windows local machine via Vagrant link
⏳ WIP Windows local machine via VirtualBox -
✔️ Completed Windows remote machine at SURF HPC Cloud link

Security

A warning from GitHub for self-hosted runners in combination with public repositories is shown here. Please take this seriously. It basically means that the combination of a self-hosted runner and a public GitHub repository is unsafe. However, there was a recent discussion indicating that GitHub may add features to make this combination safe in the near future.

Documentation for developers

If you want to check if the links in your markdown work, install markdown-link-check

npm install

then run

find . -name '*.md' -not -path './node_modules/*' -exec markdown-link-check '{}' --config .mlc-config.json ';'