diff --git a/site/content/en/docs/admission-check-controllers/provisioning.md b/site/content/en/docs/admission-check-controllers/provisioning.md index f8ed94efd9..aa15ada95a 100644 --- a/site/content/en/docs/admission-check-controllers/provisioning.md +++ b/site/content/en/docs/admission-check-controllers/provisioning.md @@ -6,14 +6,26 @@ description: > An admission check controller providing kueue integration with cluster autoscaler. --- -The Provisioning Admission Check Controller is an Admission Check Controller designed to integrate Kueue with [Kubernetes cluster-autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler). Its primary function is to create [ProvisioningRequests](https://github.com/kubernetes/autoscaler/blob/4872bddce2bcc5b4a5f6a3d569111c11b8a2baf4/cluster-autoscaler/provisioningrequest/apis/autoscaling.x-k8s.io/v1beta1/types.go#L41) for the workloads holding [Quota Reservation](/docs/concepts/#quota-reservation) and keeping the [AdmissionCheckState](/docs/concepts/admission_check/#admissioncheckstate) in sync. +The Provisioning AdmissionCheck Controller is an AdmissionCheck Controller designed to integrate Kueue with [Kubernetes cluster-autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler). Its primary function is to create [ProvisioningRequests](https://github.com/kubernetes/autoscaler/blob/4872bddce2bcc5b4a5f6a3d569111c11b8a2baf4/cluster-autoscaler/provisioningrequest/apis/autoscaling.x-k8s.io/v1beta1/types.go#L41) for the workloads holding [Quota Reservation](/docs/concepts/#quota-reservation) and keeping the [AdmissionCheckState](/docs/concepts/admission_check/#admissioncheckstate) in sync. -The controller is part of kueue. You can enable it by setting the `ProvisioningACC` feature gate. Check the [Installation](/docs/installation/#change-the-feature-gates-configuration) guide for details on feature gate configuration. +The controller is part of Kueue. You can enable it by setting the `ProvisioningACC` feature gate. Check the [Installation](/docs/installation/#change-the-feature-gates-configuration) guide for details on feature gate configuration. The Provisioning Admission Check Controller is supported on [Kubernetes cluster-autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler) versions 1.29 and later. However, some cloud-providers may not have an implementation for it. -## Parameters +## Usage +To use the Provisioning AdmissionCheck, create an [AdmissionCheck](docs/concepts/admission_check) +with `kueue.x-k8s.io/provisioning-request` as a `.spec.controllerName` and create a ProvisioningRequest configuration usign a `ProvisioningRequestConfig` object. See an example below. + +## ProvisioningRequest configuration + +There are two ways to configure ProvisioningRequest sent for your Jobs. + +- **ProvisioningRequestConfig:** This configuration in the AdmissionCheck applies to all the jobs that go through this check. +It enables you to set `provisioningClassName`, `managedResources`, and `parameters` +- **Job annotation**: This configuration applies only to a specific job. It enables you to set `parameters`, and if both the annotation and the ProvisioningRequestConfig refer same one, annotation overrides it. + +### ProvisioningRequestConfig This controller uses a `ProvisioningRequestConfig` as parameters, like: ```yaml @@ -33,6 +45,12 @@ Where: Check the [API definition](https://github.com/kubernetes-sigs/kueue/blob/main/apis/kueue/v1beta1/provisioningrequestconfig_types.go) for more details. +### Job annotation + +Another way to pass ProvisioningRequest's [parameters](https://github.com/kubernetes/autoscaler/blob/0130d33747bb329b790ccb6e8962eedb6ffdd0a8/cluster-autoscaler/apis/provisioningrequest/autoscaling.x-k8s.io/v1beta1/types.go#L115) is by using Job annotations. Every annotation with the ***provreq.kueue.x-k8s.io/*** prefix will be directly passed to created ProvisioningRequest. E.g. `provreq.kueue.x-k8s.io/ValidUntilSeconds: "60"` will pass `ValidUntilSeconds` parameter with the value of `60`. See more examples below. + +Once Kueue creates a ProvisioningRequest for the job you submitted, modifying the value of annotations in the job will have no effect in the ProvisioningRequest. + ## Example ### Setup diff --git a/site/static/examples/provisioning/sample-job.yaml b/site/static/examples/provisioning/sample-job.yaml index cb707d5288..9d36dde4b0 100644 --- a/site/static/examples/provisioning/sample-job.yaml +++ b/site/static/examples/provisioning/sample-job.yaml @@ -5,6 +5,8 @@ metadata: namespace: default labels: kueue.x-k8s.io/queue-name: user-queue + annotations: + provreq.kueue.x-k8s.io/maxRunDurationSeconds: "600" spec: parallelism: 3 completions: 3