Merge branch 'release/2.7.0'

Makefile

@@ -256,6 +256,10 @@ open-project-docs :
 	open project-docs/_build/html/index.html || \
         xdg-open project-docs/_build/html/index.html
 
+clean-project-docs:
+	rm -rf project-docs/venv
+	rm -rf project-docs/_build
+
 deploy-workshop:
 	kubectl apply -f https://github.com/vmware-tanzu-labs/lab-k8s-fundamentals/releases/download/5.0/workshop.yaml
 	kubectl apply -f https://github.com/vmware-tanzu-labs/lab-k8s-fundamentals/releases/download/5.0/trainingportal.yaml
@@ -283,6 +287,7 @@ prune-builds:
 	rm -rf training-portal/venv
 	rm -rf client-programs/bin
 	rm -rf client-programs/pkg/renderer/files
+	rm -rf project-docs/venv
 	rm -rf project-docs/_build
 
 prune-registry:

NOTICE

@@ -1,4 +1,4 @@
-Copyright 2020-2023 The Educates Authors.
+Copyright 2020-2024 The Educates Authors.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

client-programs/go.mod

@@ -147,7 +147,7 @@ require (
 	golang.org/x/time v0.5.0 // indirect
 	golang.org/x/tools v0.16.1 // indirect
 	google.golang.org/appengine v1.6.8 // indirect
-	google.golang.org/protobuf v1.31.0 // indirect
+	google.golang.org/protobuf v1.33.0 // indirect
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 	k8s.io/cli-runtime v0.29.0 // indirect

client-programs/go.sum

@@ -449,8 +449,8 @@ google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzi
 google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
 google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
-google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
-google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
+google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
+google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20200902074654-038fdea0a05b/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=

project-docs/conf.py

@@ -18,7 +18,7 @@
 # -- Project information -----------------------------------------------------
 
 project = 'Educates'
-copyright = '2020-2023 The Educates Authors'
+copyright = '2020-2024 The Educates Authors'
 author = 'Graham Dumpleton'
 
 

project-docs/custom-resources/workshop-definition.md

@@ -566,6 +566,7 @@ spec:
 
 Note that the ability to override environment variables using this field should be limited to cases where they are required for the workshop. If you want to set or override an environment for a specific workshop environment, use the ability to set environment variables in the ``WorkshopEnvironment`` custom resource for the workshop environment instead.
 
+(overriding-the-memory-available)=
 Overriding the memory available
 -------------------------------
 
@@ -629,6 +630,7 @@ In addition to secrets and configmaps these can be used to mount different types
 
 Note that ``volumeMounts`` are only added to the main workshop container. If mounting of a volume into a side car container was necessary for some purpose, then ``patches`` would need to be used to apply a patch against the complete workshop pod spec.
 
+(resource-budget-for-namespaces)=
 Resource budget for namespaces
 ------------------------------
 
@@ -952,6 +954,7 @@ This can be added after an existing ``ytt`` or ``helmTemplate`` section under ``
 
 The purpose of the overlay is to set the owner of all resources generated to be the ``WorkshopSession`` resource for the workshop session. This will ensure that any resources will be automatically deleted when the workshop session is deleted, without relying on ``kapp-controller`` to clean them up.
 
+(overriding-default-rbac-rules)=
 Overriding default RBAC rules
 -----------------------------
 
@@ -1047,6 +1050,7 @@ spec:
 
 Note that previously one would patch the workshop pod template and set ``automountServiceAccountToken`` to ``false``. That method no longer works as how the access token is mounted into the workshop container is now handled differently.
 
+(running-user-containers-as-root)=
 Running user containers as root
 -------------------------------
 
@@ -1066,6 +1070,7 @@ spec:
 
 This setting applies to the primary session namespace and any secondary namespaces that may be created.
 
+(creating-additional-namespaces)=
 Creating additional namespaces
 ------------------------------
 
@@ -1740,6 +1745,7 @@ spec:
 
 As you don't have access to the workshop files in the init container, you will need to either use a custom container image, or inject a script into the init container using a secret and a volume mount.
 
+(patching-workshop-deployment)=
 Patching workshop deployment
 ----------------------------
 

project-docs/index.rst

@@ -45,6 +45,12 @@ Educates
   workshop-content/building-an-image
   workshop-content/working-on-content
 
+.. toctree::
+  :maxdepth: 2
+  :caption: Production Guides:
+
+  production-guides/review-guidelines
+
 .. toctree::
   :maxdepth: 2
   :caption: Custom Resources:

project-docs/installation-guides/configuration-settings.md

@@ -309,6 +309,7 @@ eth0      Link encap:Ethernet  HWaddr 02:42:AC:11:00:07
 
 If the ``MTU`` size is less than 1400, then use the value given, or a smaller value, for the ``dockerd.mtu`` setting.
 
+(image-registry-pull-through-cache)=
 Image registry pull through cache
 ---------------------------------
 

project-docs/production-guides/review-guidelines.rst

@@ -0,0 +1,21 @@
+Review Guidelines
+=================
+
+Before a workshop is deployed in a production environment for others to use it should be reviewed to ensure it follows best practices, will not consume excessive amounts of resources, and does not create any undue security risks.
+
+The follow guidelines are intended to help reviewers ensure that the workshop is ready for deployment.
+
+.. toctree::
+  :maxdepth: 2
+
+  review-guidelines/changes-to-the-security-policy
+  review-guidelines/disabling-kubernetes-access
+  review-guidelines/workshop-user-admin-access
+  review-guidelines/workshop-container-memory
+  review-guidelines/workshop-container-cpu
+  review-guidelines/workshop-container-storage
+  review-guidelines/namespace-resource-budget
+  review-guidelines/workshop-container-startup
+  review-guidelines/docker-resource-requirements
+  review-guidelines/docker-container-image-registry
+  review-guidelines/hosting-images-on-docker-hub

project-docs/production-guides/review-guidelines/changes-to-the-security-policy.md

@@ -0,0 +1,32 @@
+Changes to the security policy
+==============================
+
+By default RBAC is set up for workshop session namespaces such that containers can only run as non root users due to the increased security risk of allowing workshop users to run anything as root.
+
+In an ideal world this would be fine, but too many container images out there don't follow best practices of running as a non root user and instead will only work if run as root. This is especially the case for official images from Docker Hub.
+
+As an example, the `nginx` image on Docker Hub, which is often used in example deployments for Kubernetes, will only run correctly if run as root.
+
+Workshops can override the default `restricted` security policy applied through RBAC by setting the `session.namespaces.security.policy` property. Currently this can be [set](running-user-containers-as-root) to the alternative of `baseline` to allow containers to be run as root.
+
+```yaml
+spec:
+  session:
+    namespaces:
+      security:
+        policy: baseline
+```
+
+Overall the recommended solution is not to use any container image that requires it be run as root. In the case of `nginx` one can use the `bitnami/nginx` image instead. The Bitnami catalog provides various other images as well which can run as a non root user where the official Docker Hub images will only run as root.
+
+If absolutely necessary a workshop definition can specify its own pod security policies and bind those to a distinct service account used by a deployment to give more elevated privileges, including running as a privileged container, but this should be avoided due to the huge security risks with workshop users being able to create their own deployments using the service account and a malicious container image which could compromise the cluster.
+
+**Recommendations**
+
+* If possible workshops should never use container images that require they be run as root.
+* Ensure that the security policy is not overridden to allow containers images to run as root if there is no need for it.
+* Ensure that workshops don't give themselves elevated privileges outside of the `restricted` and `baseline` defaults, especially the ability to run privileged containers, due to the extreme security risks.
+
+**Related Issues**
+
+Note that when a Kubernetes virtual cluster is enabled for use with a workshop session, the security policy for the session namespace is automatically changed from `restricted` to `baseline`. This is necessary for the virtual cluster to run properly, but also means that any deployments made to the virtual cluster by a workshop user can run as the root user.

project-docs/production-guides/review-guidelines/disabling-kubernetes-access.md

@@ -0,0 +1,21 @@
+Disabling Kubernetes access
+===========================
+
+By default, a workshop session always provides access to a namespace in the Kubernetes cluster for that session. This is so that as part of a workshop deployments can be made into the Kubernetes cluster. Usually a workshop user would only have access to the single namespace created for that session.
+
+If the topic of a workshop is such that there is never a need for a user to be able to deploy anything to the Kubernetes cluster themselves, access to the Kubernetes REST API can be disabled for the workshop.
+
+This is done by [setting](blocking-access-to-kubernetes) `session.namespaces.security.token.enabled` to `false` in the workshop definition, and results in the service account token not being mounted into the workshop container.
+
+```yaml
+spec:
+  session:
+    namespaces:
+      security:
+        token:
+          enabled: false
+```
+
+**Recommendations**
+
+* Ensure that access to the Kubernetes REST API is disabled if access is not required.

project-docs/production-guides/review-guidelines/docker-container-image-registry.md

@@ -0,0 +1,24 @@
+Docker container image registry
+===============================
+
+A workshop can enable the deployment of a container image registry per workshop session. This can be used to store container images built as part of the workshop instructions using `docker build`, `pack`, `kpack`, `kaniko` etc. The container images in the per session container image registry can then be deployed into the Kubernetes cluster.
+
+The container image registry uses a distinct deployment separate to the deployment of the workshop instance for the session. To stores images the per session container image registry is by default given a 5Gi persistent volume. The amount of memory set aside for the container image registry defaults to 768Mi. The amount of storage and memory can
+be [overridden](enabling-session-image-registry) by setting properties under `session.applications.registry` in the workshop definition.
+
+```yaml
+spec:
+  session:
+    applications:
+      registry:
+        enabled: true
+        memory: 1Gi
+        storage: 20Gi
+```
+
+**Recommendations**
+
+* Ensure that adequate memory is allocated to the per session container image registry.
+* Ensure that adequate storage space is allocated to the per session container image registry.
+* Ensure that the workshop doesn't encourage an ongoing repetitive process of building container images and pushing them to the container image registry as this will incrementally keep using more storage as no pruning is done of old image layers.
+* If possible, rather than have deployments in the cluster directly reference container images on Docker Hub, have a user pull images from Docker Hub and push them to the per workshop session image registry and deploy the container image from that registry. At the same time, ensure that use of a Docker Hub mirror for each workshop environment is configured for Educates to avoid issues with possible rate limiting by Docker Hub. Alternatively look at using a shared OCI image cache with specific workshop environments to mirror required images.

project-docs/production-guides/review-guidelines/docker-resource-requirements.md

@@ -0,0 +1,46 @@
+Docker resource requirements
+============================
+
+When docker support is enabled a dockerd instance (`dind` - docker in docker) is run as a side car container to the workshop container. When building and/or running docker images within the workshop, this is done inside of the context of this side car container and not in the main workshop container. This is the same if using tools such as `pack` which although it doesn't use the docker command, does connect to the dockerd instance running in the side car container to do the actual work.
+
+To ensure that building and/or running container images doesn't impact on the memory availability for the workshop container, the dockerd side car is given its own separate memory allocation. By default the amount of memory is 768Mi and this memory amount is guaranteed. If running docker builds which require a lot of memory (Java and NodeJS builds), you may need to increase the memory given to the dockerd side car container. This can be [overridden](enabling-ability-to-use-docker) in the workshop definition using the `session.applications.docker.memory` setting.
+
+```yaml
+spec:
+  session:
+    applications:
+      docker:
+        enabled: true
+        memory: 1Gi
+```
+
+Because of possible large transient filesystem space requirements when building container images, as well as the space required to store the built container image, or any container image pulled down into the local dockerd instance, a persistent volume is always allocated for the dockerd container. This avoids the problem that you might run out of space on the Kubernetes node were lots of workshop instances running at the same time, but depending on how large the container images are, or how much transient space is required to build a container image, you may need to increase the amount of storage. By default the amount of storage given to dockerd is 5Gi. This can be [overridden](enabling-ability-to-use-docker) in the workshop definition using the `session.applications.docker.storage` setting.
+
+```
+spec:
+  session:
+    applications:
+      docker:
+        enabled: true
+        storage: 20Gi
+```
+
+**Recommendations**
+
+* Ensure that docker support is not enabled if not being used because of additional resource requirements but also the security risks implicit in allowing dockerd to be run.
+* Ensure that adequate memory is allocated to the dockerd side car container.
+* Ensure that adequate storage space is allocated to the dockerd side car container.
+* Ensure that the number of container images pulled down using docker is limited to what is required.
+* Ensure that you don't encourage running successive builds as each change will result in more container layers being stored.
+* Ensure that users are asked to delete container images to free up storage space if they can, before proceeding with subsequent workshop steps, rather than just allocating more storage space.
+* Ensure that users are asked to delete stopped containers to free up storage space if they can, before proceeding with subsequent workshop steps, rather than just allocating more storage space. Alternatively, use the `-rm` option to `docker run` to ensure that stopped containers are automatically deleted when they exit.
+
+**Related Issues**
+
+Note that memory and storage requirements for the dockerd side car container should be added to the memory and storage requirements of the main workshop container when determining resource requirements for each workshop instance. In the case of memory, since both containers are in the same pod, the combined memory requirement has to be available on a node before a workshop instance can be scheduled to the node. For storage, a separate persistent volume claim is used for the dockerd side car container, but because it is part of the same pod as the main workshop container, a node has to have available with enough persistent volume mount points to support the combined total number of persistent volumes used.
+
+Note that you cannot use storage space allocations that may work on local Kubernetes clusters using Kind or Minikube as a guide for how much you should use as the amount of storage requested in the case of those Kubernetes clusters is ignored and instead you can always use up to as much space as the VM or host system in which the Kubernetes cluster is running has for file system space.
+
+Note that enabling docker support is always a risk and the Kubernetes cluster can be readily compromised by a knowledgeable person intent on mischief because dockerd has to be run using a privileged container with the user then also being able to run containers using docker as privileged. If possible avoid using docker and use methods for building container images that don't rely on docker, such as kaniko and Java native systems for building container images.
+
+ If the only reason for using docker is to run some services for each workshop session, instead try and run them as Kubernetes services. Alternatively, use the mechanism of the docker support to run services using a docker-compose configuration snippet. Using this method will by default result in the docker socket not being exposed inside of the workshop container thus reducing the security risk.