Racetrack Plugin: Remote Kubernetes Infrastructure

A Racetrack plugin allowing to deploy services to remote Kubernetes (running on different host)

Setup

1. Install racetrack client and generate ZIP plugin by running:

   make bundle

   Afterward, activate the plugin in Racetrack Dashboard Admin page by uploading the zipped plugin file:

   racetrack plugin install remote-kubernetes-*.zip

   Alternatively, you can install the latest plugin by running:

   racetrack plugin install github.com/TheRacetrack/plugin-remote-kubernetes

2. Build image of remote Pub gateway, suitable for kubernetes setup.

   make build-remote-pub

   This will produce ghcr.io/theracetrack/plugin-remote-kubernetes/pub-remote:latest image. Now retag it and push it to the registry that is accessible by your Kubernetes cluster (make sure it can pull from there):

   TARGET_IMAGE=localhost:5000/theracetrack/plugin-remote-kubernetes/pub-remote:latest
   
   docker tag ghcr.io/theracetrack/plugin-remote-kubernetes/pub-remote:latest $TARGET_IMAGE
   docker push $TARGET_IMAGE

3. Deploy Racetrack's PUB gateway on a remote host, which will dispatch the traffic to the local jobs. Generate a strong password that will be used as a token to authorize only the requests coming from the main Racetrack:

   REMOTE_GATEWAY_TOKEN='5tr0nG_PA55VoRD'
   TARGET_IMAGE=kind-registry:5000/theracetrack/plugin-remote-kubernetes/pub-remote:latest
   NAMESPACE=racetrack
   
   cat << EOF | kubectl apply -f -
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     namespace: $NAMESPACE
     name: pub
     labels:
       app.kubernetes.io/name: pub
   spec:
     replicas: 1
     selector:
       matchLabels:
         app.kubernetes.io/name: pub
     template:
       metadata:
         labels:
           app.kubernetes.io/name: pub
       spec:
         serviceAccountName: null
         securityContext:
           supplementalGroups: [200000]
           fsGroup: 200000
           runAsUser: 100000
           runAsGroup: 100000
         automountServiceAccountToken: true
         priorityClassName: high-priority
         hostname: pub
         subdomain: pub
         containers:
           - name: pub
             image: $TARGET_IMAGE
             imagePullPolicy: Always
             ports:
               - containerPort: 7005
             tty: true
             env:
               - name: PUB_PORT
                 value: '7005'
               - name: AUTH_REQUIRED
                 value: 'true'
               - name: REMOTE_GATEWAY_MODE
                 value: 'true'
               - name: REMOTE_GATEWAY_TOKEN
                 value: '$REMOTE_GATEWAY_TOKEN'
             livenessProbe:
               httpGet:
                 path: /live
                 port: 7005
               initialDelaySeconds: 30
               periodSeconds: 10
             readinessProbe:
               httpGet:
                 path: /ready
                 port: 7005
               initialDelaySeconds: 3
               periodSeconds: 10
             volumeMounts:
               - mountPath: /.kube
                 name: tmp-k8s-volume
         volumes:
           - name: tmp-k8s-volume
             emptyDir:
               sizeLimit: 1Gi
   ---
   apiVersion: v1
   kind: Service
   metadata:
     namespace: $NAMESPACE
     name: pub
     labels:
       app.kubernetes.io/name: pub
   spec:
     selector:
       app.kubernetes.io/name: pub
     type: NodePort
     ports:
       - name: pub
         nodePort: 30005
         port: 7005
         targetPort: 7005
   EOF

   Make sure pods can speak to local Kubernetes API inside the cluster.

4. Go to Racetrack's Dashboard, Administration, Edit Config of the plugin. Prepare the following data:

   • IP or DNS hostname of your cluster
   • Credentials to the Docker Registry, where Job images will be located.

   Save the YAML configuration of the plugin:

   infrastructure_targets:
     remote-k8s:
       remote_gateway_url: 'http://1.2.3.4:7105/pub'
       remote_gateway_token: '5tr0nG_PA55VoRD'
       job_k8s_namespace: 'racetrack'
   
   docker: 
     docker_registry: 'docker.registry.example.com'
     username: 'DOCKER_USERNAME'
     password: 'READ_WRITE_TOKEN'