v1.95.2

[gardener/gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed which prevented Plutono dashboards contributed from extensions to appear in the UI. by @rfranzke [#9810]
• [OPERATOR] A race condition has been fixed which could cause unrelated Pods to claim the PersistentVolume of a Prometheus or Alertmanager deployment during migration to the management of prometheus-operator. by @rfranzke [#9840]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.95.2
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.95.2
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.95.2
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.95.2
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.95.2
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.95.2
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.95.2
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.95.2

v1.95.1

[gardener/gardener]

🏃 Others

• [OPERATOR] gardenlet: An issue causing the blackbox-exporter Deployment to be created and to be unhealthy in the Shoot control plane for Shoots with .spec.purpose=testing is now fixed. by @ialidzhikov [#9798]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.95.1
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.95.1
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.95.1
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.95.1
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.95.1
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.95.1
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.95.1
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.95.1

v1.94.2

[gardener/gardener]

🐛 Bug Fixes

• [USER] A bug has has been fixed which caused unneeded gardener-node-agent reconciliations after each Shoot reconciliation even if the underlying OperatingSystemConfig did not contain relevant changes. by @rfranzke [#9731]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.94.2
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.94.2
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.94.2
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.94.2
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.94.2
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.94.2
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.94.2
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.94.2

v1.95.0

[gardener/gardener]

⚠️ Breaking Changes

• [OPERATOR] The .monitoring.shoot.remoteWrite.queueConfig field is no longer available in the gardenlet component configuration. If needed, you have to register a webhook for the monitoring.coreos.com/v1.Prometheus object named shoot in the shoot namespaces. The webhook can inject the needed configuration in .spec.remoteWrite[0].queueConfig. by @rfranzke [#9695]

📰 Noteworthy

• [DEVELOPER] The extensions.gardener.cloud/v1alpha1.Worker resource now has a new .spec.pools[].userDataSecretRef field which references a Secret containing the actual user data. the .spec.pools[].userData field is deprecated and will be removed in a future version. Worker extensions should fetch the user data from the secret and can use the extensions/pkg/controller/worker.FetchUserData helper function for it. by @rfranzke [#9722]
• [DEVELOPER] The legacy method for extensions to provide observability configuration for shoot clusters (via ConfigMaps labelled with extensions.gardener.cloud/configuration=monitoring) is deprecated and will be removed in a future release. Please refer to this document to get information about the new, recommended way, and start migrating to it. by @rfranzke [#9695]

✨ New Features

• [OPERATOR] Gardener can now support clusters with Kubernetes version 1.30. To allow creation/update of 1.30 clusters you will have to update the version of your provider extension(s) to a version that supports 1.30 as well. Please consult the respective releases and notes in the provider extension's repository. by @shafeeqes [#9689]
• [OPERATOR] A new feature gate named VPAAndHPAForAPIServer is introduced to gardenlet. When enabled, the Shoot Kubernetes API Server is scaled simultaneously by VPA and HPA on the same metric (CPU and memory usage). The new feature aims to replace the existing HVPA autoscaling mechanism for the Shoot Kubernetes API server. by @ialidzhikov [#9678]
• [USER] It is now possible to configure Projects with the "four-👀 approval concept for deletion" concept. For now, this can only be applied to Shoots. If configured, the user confirming a Shoot deletion (via the confirmation.gardener.cloud/deletion annotation) must not be the same user who is sending the DELETE request. This can help preventing accidental/unintentional Shoot deletion. Find all information about the feature in this document. by @rfranzke [#9680]
• [DEVELOPER] Gardener can now support clusters with Kubernetes version 1.30. Extension developers have to prepare individual extensions as well to work with 1.30. by @shafeeqes [#9689]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed which caused regeneration of managedresource-shoot-core-system-* Secrets on each Shoot reconciliation. by @rfranzke [#9718]
• [USER] A bug has has been fixed which caused unneeded gardener-node-agent reconciliations after each Shoot reconciliation even if the underlying OperatingSystemConfig did not contain relevant changes. by @rfranzke [#9723]

🏃 Others

• [OPERATOR] e2e-kind tests can now run successfully in an IPv4-only environment by @ScheererJ [#9693]
• [OPERATOR] Validation of DNSRecords: allow domain names starting with an underscore "_" by @MartinWeindel [#9714]
• [OPERATOR] The istio ingress gateway access log now includes the connections initiated via apiserver-proxy, i.e. cluster-internal communication via kubernetes.default.svc.cluster.local. by @ScheererJ [#9686]
• [OPERATOR] Replaced HVPA for the vali StatefulSet with VPA. Additionally, the curator kube-rbac-proxy and telegraf containers of the vali StatefulSet now specify CPU resource requests of 5m each. by @plkokanov [#9611]
• [OPERATOR] Updated MCM metrics list used to configure prometheus by @rishabh-11 [#9684]
• [OPERATOR] The kube-controller-manager component is now scaled by VPA, instead of HVPA. by @andrerun [#9698]
• [OPERATOR] Modified the CPU and memory resource requests for the plutono container to 5m and 45Mi, respectively. Additionally, reduced the vali container CPU resource requests to 20m. by @plkokanov [#9754]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.95.0
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.95.0
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.95.0
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.95.0
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.95.0
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.95.0
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.95.0
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.95.0

v1.93.1

[gardener/gardener]

🐛 Bug Fixes

• [USER] A bug has has been fixed which caused unneeded gardener-node-agent reconciliations after each Shoot reconciliation even if the underlying OperatingSystemConfig did not contain relevant changes. by @rfranzke [#9732]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.93.1
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.93.1
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.93.1
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.93.1
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.93.1
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.93.1
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.93.1
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.93.1

v1.92.3

[gardener/gardener]

🐛 Bug Fixes

• [USER] A bug has has been fixed which caused unneeded gardener-node-agent reconciliations after each Shoot reconciliation even if the underlying OperatingSystemConfig did not contain relevant changes. by @rfranzke [#9733]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.92.3
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.92.3
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.92.3
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.92.3
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.92.3
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.92.3
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.92.3
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.92.3

v1.94.1

[gardener/gardener]

🐛 Bug Fixes

• [OPERATOR] Fix an issue in the etcd component which caused Shoot deletion to fail when the VPAForETCD feature gate was enabled by @voelzmo [#9703]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.94.1
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.94.1
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.94.1
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.94.1
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.94.1
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.94.1
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.94.1
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.94.1

v1.94.0

[gardener/gardener]

📰 Noteworthy

• [OPERATOR] Five minutes Infrastructure Cleanup Wait Period during shoot deletion was removed. Shoot annotation shoot.gardener.cloud/infrastructure-cleanup-wait-period-seconds which could be used to configure this period was removed, too. by @oliver-goetz [#9632]
• [DEVELOPER] The tools installed via the tools.mk make file are now by default installed in an OS and arch specific folder to allow running make targets from different platforms sharing the same source code.
  The previous behavior can be achieved by setting the variable TOOLS_BIN_DIR to hack/tools/bin to any make target. by @vpnachev [#9589]
• [DEVELOPER] Today's method of providing Plutono dashboards for garden or shoot clusters is deprecated and will be removed in a future release. Migrate to the new approach (see this document) for details. by @rfranzke [#9624]

✨ New Features

• [OPERATOR] gardener-operator is now managing the Gardener Dashboard web terminal controller manager when .spec.virtualCluster.gardener.gardenerDashboard.terminal is set in the Garden resource. Read more about it here by @rfranzke [#9646]
• [OPERATOR] gardener-node-agent no longer watches all Nodes in the cluster but restricts to only the Node it is responsible for (with the help of label/field selectors). This should lead to a significant reduction of network I/O, especially for shoot clusters with many nodes. by @rfranzke [#9672]
• [OPERATOR] gardener-operator now deploys two more Prometheus replicas into the garden namespace for storing long-term metrics. Read more about it here. by @rfranzke [#9606]
• [OPERATOR] A new feature gate named VPAForETCD is now introduced for gardenlet and gardener-operator. When enabled, VPA for etcd is used, regardless of the HVPA feature gate setting. The new VPA limits scaling down to a Shoot's maintenance window or even entirely based on the ShootClass in the same way as it is currently done for HVPA. by @voelzmo [#8984]
• [OPERATOR] gardener-operator is now managing the Gardener Dashboard when .spec.virtualCluster.gardener.gardenerDashboard is set in the Garden resource. Read more about it here by @rfranzke [#9583]
• [USER] It is now possible to define a higher number of maximum worker count in a shoot than pods and nodes networks allow. cluster-autoscaler ensures that not more nodes than the networking settings allow will be created. by @oliver-goetz [#9599]

🐛 Bug Fixes

• [OPERATOR] gardener-operator is now capable of reconciling shoot cluster-specific NetworkPolicys in case the garden cluster is a seed cluster at the same time. by @rfranzke [#9658]
• [OPERATOR] Fixed prometheus alerting rules for Seeds with unhealthy control-planes by @voelzmo [#9692]
• [OPERATOR] In the migrate flow of control plane migration the Deleting extensions before kube-apiserver task now depends on the Waiting until extension resources have been deleted task. by @plkokanov [#9651]
• [OPERATOR] Only update network policy allow-to-runtime-apiserver after resolver has been synced. by @MartinWeindel [#9644]

🏃 Others

• [OPERATOR] Updated VPA to 1.1.1 by @voelzmo [#8984]
• [OPERATOR] If a previous file copy attempt failed gardener-node-agent now deletes leftover *.tmp files instead of returning an error. by @oliver-goetz [#9630]
• [OPERATOR] extension library: An issue causing the backup.gardener.cloud/created-by annotation not being added on existing etcd-backup Secrets is now fixed. by @ialidzhikov [#9613]
• [OPERATOR] Added a cleanup function to gardenlet which is executed at startup and deletes orphaned VPAs with label role: vali-vpa that were previously managed by the HVPA deployed for vali. by @plkokanov [#9681]
• [OPERATOR] The gardenlet now runs as nonroot user and group 65532. by @AleksandarSavchev [#9669]
• [OPERATOR] A new plutono dashboard named Resource usage by container is added to garden/plutono. It shows aggregated CPU/memory usage vs requests/limits and utilization per container (currently only metrics for kube-apiserver containers are federated). by @ialidzhikov [#9643]
• [OPERATOR] Containers, configured to run as non-root, are now validated to start with non-root user by the kubelet. by @AleksandarSavchev [#9640]
• [OPERATOR] The fluent-operator component now runs as nonroot user and group 65532. by @AleksandarSavchev [#9640]
• [OPERATOR] The kube-controller-manager's (H)VPA minAllowed memory is reduced from 100Mi to 50Mi. The kube-apiserver's HVPA minAllowed memory is reduced from 400M to 200M. by @ialidzhikov [#9654]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.94.0
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.94.0
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.94.0
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.94.0
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.94.0
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.94.0
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.94.0
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.94.0

v1.93.0

[gardener/gardener]

📰 Noteworthy

• [OPERATOR] Set kube-apiserver maxReplicas=3 for all Shoots that are not annotated with alpha.control-plane.scaling.shoot.gardener.cloud/scale-down-disabled=true. by @voelzmo [#9605]

✨ New Features

• [OPERATOR] gardener-operator now deploys two Prometheus replicas into the garden namespace. Read more about it here. by @rfranzke [#9543]
• [OPERATOR] A new gardenlet feature gate called ShootManagedIssuer was introduced. This feature gate guards the functionality described in GEP-24 until all of the components mentioned in the enhancement proposal are implemented by Gardener. by @dimityrmirchev [#9489]
• [OPERATOR] A new admission plugin ShootResourceReservation has been added to gardener-apiserver. It supports calculating resource reservations (memory/CPU/PID) for the kubelet.kubeReserved fields in Shoots based on the available resources of a machine type. This only applies when useGKEFormula is set to true. Otherwise, the old static values remain to be used. by @MichaelEischer [#9449]
• [OPERATOR] Support for proxy protocol is added to the istio ingress gateway to preserve the client source IP addresses. by @DockToFuture [#9526]

🐛 Bug Fixes

• [OPERATOR] Fix kube-apiserver advertise address for ipv6 local setup. by @axel7born [#9555]
• [OPERATOR] When vali is disabled in the GardenletConfiguration its fluentbit ClusterOutputs are no longer deployed. by @maboehm [#9525]
• [OPERATOR] Istio-ingress gateway dashboard now shows the correct sent tcp traffic metric and the correct memory usage. by @ScheererJ [#9596]
• [OPERATOR] A bug in gardener-node-agent which prevented copying files between different block devices has been fixed. by @oliver-goetz [#9614]
• [USER] A bug which mounted the kubelet data volume to /var/lib instead of /var/lib/kubelet when kubeletDataVolumeName was set has been fixed. by @oliver-goetz [#9614]

🏃 Others

• [OPERATOR] The vpn-seed-server now has better minimum memory settings so that less auto-scaling should occur. by @ScheererJ [#9590]
• [OPERATOR] Resource utilization metrics for the kube-apiserver container are now federated in the runtime/prometheus. by @ialidzhikov [#9581]
• [OPERATOR] K8s dashboard tests are classified as beta. by @hendrikKahl [#9567]
• [OPERATOR] Update Istio to v1.21.1 by @axel7born [#9560]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.93.0
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.93.0
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.93.0
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.93.0
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.93.0
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.93.0
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.93.0
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.93.0

v1.92.2

[gardener/gardener]

🐛 Bug Fixes

• [USER] A bug which mounted the kubelet data volume to /var/lib instead of /var/lib/kubelet when kubeletDataVolumeName was set has been fixed. by @oliver-goetz [#9615]
• [OPERATOR] A bug in gardener-node-agent which prevented copying files between different block devices has been fixed. by @oliver-goetz [#9615]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.92.2
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.92.2
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.92.2
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.92.2
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.92.2
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.92.2
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.92.2
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.92.2