title | date | weight | description |
---|---|---|---|
Run A JobSet |
2023-06-16 |
7 |
Run a Kueue scheduled JobSet.
|
This document explains how you can use Kueue’s scheduling and resource management functionality when running JobSet Operator JobSets.
This guide is for batch users that have a basic understanding of Kueue. For more information, see Kueue's overview.
-
Check Administer cluster quotas for details on the initial Kueue setup.
-
See JobSet Installation for installation and configuration details of JobSet Operator.
When running JobSets on Kueue, take into consideration the following aspects:
The target local queue should be specified in the metadata.labels
section of the JobSet configuration.
metadata:
labels:
kueue.x-k8s.io/queue-name: user-queue
If the JobSet is submitted to a queue using MultiKueue one additional label needs to be specified.
alpha.jobset.sigs.k8s.io/managed-by: kueue
The resource needs of the workload can be configured in the spec.replicatedJobs
. Should also be taken into account that number of replicas, parallelism and completions affect the resource calculations.
- replicas: 1
template:
spec:
completions: 2
parallelism: 2
template:
spec:
containers:
- resources:
requests:
cpu: 1
The first PriorityClassName of spec.replicatedJobs
that is not empty will be used as the priority.
- template:
spec:
template:
spec:
priorityClassName: high-priority
The JobSet looks like the following:
# jobset-sample.yaml
apiVersion: jobset.x-k8s.io/v1alpha2
kind: JobSet
metadata:
generateName: sleep-job-
labels:
kueue.x-k8s.io/queue-name: user-queue
spec:
network:
enableDNSHostnames: false
subdomain: some-subdomain
replicatedJobs:
- name: workers
replicas: 2
template:
spec:
parallelism: 4
completions: 4
backoffLimit: 0
template:
spec:
containers:
- name: sleep
image: busybox
resources:
requests:
cpu: 1
memory: "200Mi"
command:
- sleep
args:
- 100s
- name: driver
template:
spec:
parallelism: 1
completions: 1
backoffLimit: 0
template:
spec:
containers:
- name: sleep
image: busybox
resources:
requests:
cpu: 2
memory: "200Mi"
command:
- sleep
args:
- 100s
# jobset-sample.yaml
apiVersion: jobset.x-k8s.io/v1alpha2
kind: JobSet
metadata:
generateName: sleep-job-
labels:
alpha.jobset.sigs.k8s.io/managed-by: kueue
kueue.x-k8s.io/queue-name: user-queue
spec:
network:
enableDNSHostnames: false
subdomain: some-subdomain
replicatedJobs:
- name: workers
replicas: 2
template:
spec:
parallelism: 4
completions: 4
backoffLimit: 0
template:
spec:
containers:
- name: sleep
image: busybox
resources:
requests:
cpu: 1
memory: "200Mi"
command:
- sleep
args:
- 100s
- name: driver
template:
spec:
parallelism: 1
completions: 1
backoffLimit: 0
template:
spec:
containers:
- name: sleep
image: busybox
resources:
requests:
cpu: 2
memory: "200Mi"
command:
- sleep
args:
- 100s
You can run this JobSet with the following commands:
# To monitor the queue and admission of the jobs, you can run this example multiple times:
kubectl create -f jobset-sample.yaml