Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: refactor kubernetes helm chart #4480

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat: refactor kubernetes helm chart #4480

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions kubernetes/Chart.lock
View file
Open in desktop
@@ -1,6 +1,6 @@
dependencies:
- name: postgresql
repository: https://charts.bitnami.com/bitnami/
version: 12.11.1
digest: sha256:3c10008175c4f5c1cec38782f5a7316154b89074c77ebbd9bcc4be4f5ff21122
generated: "2023-09-14T22:40:43.171275362Z"
version: 14.3.1
JuniorJPDJ marked this conversation as resolved.
Show resolved Hide resolved
digest: sha256:cd32cbb4ea4af17b1d9c6bcad0218b90df16c358b1e38490a27ccf758a5abb4d
generated: "2024-03-09T04:47:50.450209116+01:00"
38 changes: 24 additions & 14 deletions kubernetes/Chart.yaml
View file
Open in desktop
@@ -1,22 +1,32 @@
apiVersion: v2
name: invidious
description: Invidious is an alternative front-end to YouTube
version: 1.1.1
appVersion: 0.20.1

type: application

version: 2.0.0
appVersion: latest

dependencies:
- name: postgresql
version: ~14.3.1
repository: "https://charts.bitnami.com/bitnami/"
condition: postgresql.enabled

engine: gotpl

keywords:
- youtube
- proxy
- video
- privacy
- youtube
- proxy
- video
- privacy

home: https://invidio.us/
icon: https://raw.githubusercontent.com/iv-org/invidious/05988c1c49851b7d0094fca16aeaf6382a7f64ab/assets/favicon-32x32.png

sources:
- https://github.com/iv-org/invidious
- https://github.com/iv-org/invidious

maintainers:
- name: Leon Klingele
email: mail@leonklingele.de
dependencies:
- name: postgresql
version: ~12.11.0
repository: "https://charts.bitnami.com/bitnami/"
engine: gotpl
- name: Leon Klingele
email: mail@leonklingele.de
42 changes: 39 additions & 3 deletions kubernetes/templates/_helpers.tpl
View file
Open in desktop
Expand Up @@ -9,8 +9,44 @@ Expand the name of the chart.
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "invidious.fullname" -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "invidious.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "invidious.labels" -}}
helm.sh/chart: {{ include "invidious.chart" . }}
{{ include "invidious.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{/*
Selector labels
*/}}
{{- define "invidious.selectorLabels" -}}
app.kubernetes.io/name: {{ include "invidious.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
11 changes: 0 additions & 11 deletions kubernetes/templates/configmap.yaml
View file
Open in desktop

This file was deleted.

106 changes: 63 additions & 43 deletions kubernetes/templates/deployment.yaml
View file
Open in desktop
Expand Up @@ -3,59 +3,79 @@ kind: Deployment
metadata:
name: {{ template "invidious.fullname" . }}
labels:
app: {{ template "invidious.name" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: {{ .Release.Name }}
{{- include "invidious.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
app: {{ template "invidious.name" . }}
release: {{ .Release.Name }}
{{- include "invidious.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
app: {{ template "invidious.name" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: {{ .Release.Name }}
{{- include "invidious.labels" . | nindent 8 }}
{{- with .Values.podLabels }}
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
enableServiceLinks: false
securityContext:
runAsUser: {{ .Values.securityContext.runAsUser }}
runAsGroup: {{ .Values.securityContext.runAsGroup }}
fsGroup: {{ .Values.securityContext.fsGroup }}
{{- toYaml .Values.podSecurityContext | nindent 8 }}
initContainers:
- name: wait-for-postgresql
image: postgres
args:
- /bin/sh
- -c
- until pg_isready -h {{ .Values.config.db.host }} -p {{ .Values.config.db.port }} -U {{ .Values.config.db.user }}; do echo waiting for database; sleep 2; done;
image: actions/pg_isready
command:
- /bin/bash
- -c
- 'until pg_isready -h "{{ .Values.config.db.host }}" -p "{{ .Values.config.db.port }}" -U "{{ .Values.config.db.user }}"; do echo waiting for database; sleep 2; done'
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- containerPort: 3000
env:
- name: INVIDIOUS_CONFIG
valueFrom:
configMapKeyRef:
key: INVIDIOUS_CONFIG
name: {{ template "invidious.fullname" . }}
securityContext:
allowPrivilegeEscalation: {{ .Values.securityContext.allowPrivilegeEscalation }}
capabilities:
drop:
- ALL
resources:
{{ toYaml .Values.resources | indent 10 }}
readinessProbe:
httpGet:
port: 3000
path: /
livenessProbe:
httpGet:
port: 3000
path: /
initialDelaySeconds: 15
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: {{ .Values.config.port }}
protocol: TCP
envFrom:
- secretRef:
name: {{ template "invidious.fullname" . }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
readinessProbe:
httpGet:
port: http
path: /
livenessProbe:
httpGet:
port: http
path: /
startupProbe:
httpGet:
port: http
path: /
periodSeconds: 5
failureThreshold: 30
restartPolicy: Always
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
4 changes: 1 addition & 3 deletions kubernetes/templates/hpa.yaml
View file
Open in desktop
Expand Up @@ -4,9 +4,7 @@ kind: HorizontalPodAutoscaler
metadata:
name: {{ template "invidious.fullname" . }}
labels:
app: {{ template "invidious.name" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: {{ .Release.Name }}
{{- include "invidious.labels" . | nindent 4 }}
spec:
scaleTargetRef:
apiVersion: apps/v1
Expand Down
61 changes: 61 additions & 0 deletions kubernetes/templates/ingress.yaml
View file
Open in desktop
@@ -0,0 +1,61 @@
{{- if .Values.ingress.enabled -}}
{{- $fullName := include "invidious.fullname" . -}}
{{- $svcPort := .Values.service.port -}}
{{- if and .Values.ingress.className (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) }}
{{- if not (hasKey .Values.ingress.annotations "kubernetes.io/ingress.class") }}
{{- $_ := set .Values.ingress.annotations "kubernetes.io/ingress.class" .Values.ingress.className}}
{{- end }}
{{- end }}
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1
{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: {{ $fullName }}
labels:
{{- include "invidious.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- if and .Values.ingress.className (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
{{- if and .pathType (semverCompare ">=1.18-0" $.Capabilities.KubeVersion.GitVersion) }}
pathType: {{ .pathType }}
{{- end }}
backend:
{{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }}
service:
name: {{ $fullName }}
port:
number: {{ $svcPort }}
{{- else }}
serviceName: {{ $fullName }}
servicePort: {{ $svcPort }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
9 changes: 9 additions & 0 deletions kubernetes/templates/secret.yaml
View file
Open in desktop
@@ -0,0 +1,9 @@
apiVersion: v1
kind: Secret
metadata:
name: {{ template "invidious.fullname" . }}
labels:
{{- include "invidious.labels" . | nindent 4 }}
stringData:
INVIDIOUS_CONFIG: |
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it better to leave INVIDIOUS_CONFIG as a configmap, with secrets being templated in as needed? Having a separate secrets.yaml would also sensitive info to actually be encrypted (K8s Secret resources aren't actually encrypted by default), for example by using the helm-secrets plugin, without making changes to INVIDIOUS_CONFIG needlessly difficult.

For example:

# secrets.yaml
postgresql:
  auth:
    username: kemal
    password: kemal
  primary:
    initdb:
      username: kemal
      password: kemal

config:
  db:
    user: kemal
    password: kemal
  hmac_key: hmac_key
# values.yaml
postgresql:
  enabled: true
  image:
    tag: 16
  auth:
    # username: SECRET
    # password: SECRET
    database: invidious
  primary:
    initdb:
      # username: SECRET
      # password: SECRET
      scriptsConfigMap: invidious-postgresql-init

config:
  db:
    # user: kemal
    # password: kemal
    host: invidious-postgresql
    port: 5432
    dbname: invidious
  # hmac_key: SECRET

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do agree on that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I wanted to do the same but I lack the idea how to do this properly. It would be easier if config options would be a separate env vars, but this is one env var and we are unable to concatenate it in helm chart before pushing it to the pod/container.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the solution here allows the usage of helm-secret without any difficulty - it's an object, not a string in the values, so you can split the secret parts to other values file and encrypt it with helm-secrets using SOPS or provide it externally using vals.
Exactly the same solution as you proposed with snippets will work now without any changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secrets aren't encrypted by default, but ConfigMaps cannot be encrypted at all.
If we have an idea how we can split part of config to go to ConfigMap and Secret we can then roll it back to ConfigMap and selectively move parts of config values to Secret. If we have no idea how to do this, I'd personally prefer to have whole INVIDIOUS_CONFIG env var in the Secret due to sensitive values inside.

Encryption of values being input to helm chart doesn't change anything here, as those values will be stored unencrypted in etcd even when someone enabled Secret encryption in the cluster due to being just a ConfigMap. Also it will be able to be accessed by applications running in the cluster with ConfigMap access, but without Secret access.

{{- toYaml .Values.config | nindent 4 }}
16 changes: 7 additions & 9 deletions kubernetes/templates/service.yaml
View file
Open in desktop
Expand Up @@ -3,18 +3,16 @@ kind: Service
metadata:
name: {{ template "invidious.fullname" . }}
labels:
app: {{ template "invidious.name" . }}
chart: {{ .Chart.Name }}
release: {{ .Release.Name }}
{{- include "invidious.labels" . | nindent 4 }}
spec:
type: {{ .Values.service.type }}
{{- with .Values.service.loadBalancerIP }}
loadBalancerIP: {{ . }}
{{- end }}
ports:
- name: http
port: {{ .Values.service.port }}
targetPort: 3000
targetPort: http
protocol: TCP
selector:
app: {{ template "invidious.name" . }}
release: {{ .Release.Name }}
{{- if .Values.service.loadBalancerIP }}
loadBalancerIP: {{ .Values.service.loadBalancerIP }}
{{- end }}
{{- include "invidious.selectorLabels" . | nindent 4 }}