From 92a00327bbaad71ff8069f4a08feafac9ed731f7 Mon Sep 17 00:00:00 2001 From: Yuki Iwai Date: Wed, 14 Feb 2024 23:38:47 +0900 Subject: [PATCH 1/7] KEP-3998: Add JobSuccessPolicy Documentation Signed-off-by: Yuki Iwai --- .../concepts/workloads/controllers/job.md | 57 +++++++++++++++++++ .../feature-gates/job-success-policy.md | 14 +++++ .../job-success-policy-example.yaml | 25 ++++++++ 3 files changed, 96 insertions(+) create mode 100644 content/en/docs/reference/command-line-tools-reference/feature-gates/job-success-policy.md create mode 100644 content/en/examples/controllers/job-success-policy-example.yaml diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index f4462477f15d7..504ce4b6f1512 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -1050,6 +1050,63 @@ after the operation: the built-in Job controller and the external controller indicated by the field value. {{< /warning >}} +### Success policy {#success-policy} + +{{< feature-state for_k8s_version="v1.29" state="alpha" >}} + +{{< note >}} +You can only configure a success policy for an Indexed Job if you have the +`JobSuccessPolicy` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) +enabled in your cluster. +{{< /note >}} + +When you run an indexed Job, a success policy defined with the `spec.successPolicy` field, +allows you to define when a Job can be declared as succeeded based on the number of succeeded pods. + +In some situations, you may want to have a better control when handling Pod +successes than the control provided by the `.spec.completins`. +There are some examples of use cases: + +* To optimize costs of running workloads by avoiding unnecessary Pod running, + you can terminate a Job as soon as one of its Pods succeeds. +* To care only about a leader index in determining the success or failure of a Job + in a batch workloads such as MPI and PyTorch etc. + +You can configure a success policy, in the `.spec.successPolicy` field, +to meet the above use cases. This policy can handle Job successes based on the +number of succeeded pods. After the Job meet success policy, the lingering Pods +are terminated by the Job controller. + +When you specify the only `.spec.successPolicy.rules[*].succeededIndexes`, +once all indexes specified in the `succeededIndexes` succeeded, the Job is marked as succeeded. +The `succeededIndexes` must be a list within 0 to `.spec.completions-1` and +must not contain duplicate indexes. The `succeededIndexes` is represented as intervals separated by a hyphen. +The number are listed in represented by the first and last element of the series, separated by a hyphen. +For example, if you want to specify 1, 3, 4, 5 and 7, the `succeededIndexes` is represented as `1,3-5,7`. + +When you specify the only `spec.successPolicy.rules[*].succeededCount`, +once the number of succeeded indexes reaches the `succeededCount`, the Job is marked as succeeded. + +When you specify both `succeededIndexes` and `succeededCount`, +once the number of succeeded indexes specified in the `succeededIndexes` reaches the `succeededCount`, +the Job is marked as succeeded. + +Note that when you specify multiple rules in the `.spec.succeessPolicy.rules`, +the rules are evaluated in order. Once the Job meets a rule, the remaining rules are ignored. + +Here is a manifest for a Job with `successPolicy`: + +{{% code_sample file="/controllers/job-success-policy-example.yaml" %}} + +In the example above, the rule of the success policy specifies that +the Job should be marked succeeded and terminate the lingering Pods +if one of the 0, 1, and 2 indexes succeeded. + +{{< note >}} +When you specify both a success policy and some terminating policies such as `.spec.backoffLimit` and `.spec.podFailurePolicy`, +once the Job meets both policies, the terminating policies are respected and a success policy is ignored. +{{< /note >}} + ## Alternatives ### Bare Pods diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates/job-success-policy.md b/content/en/docs/reference/command-line-tools-reference/feature-gates/job-success-policy.md new file mode 100644 index 0000000000000..601680357ccc9 --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates/job-success-policy.md @@ -0,0 +1,14 @@ +--- +title: JobSuccessPolicy +content_type: feature_gate + +_build: + list: never + render: false + +stages: + - stage: alpha + defaultValue: false + fromVersion: "1.30" +--- +Allow users to specify when a Job can be declared as succeeded based on the set of succeeded pods. diff --git a/content/en/examples/controllers/job-success-policy-example.yaml b/content/en/examples/controllers/job-success-policy-example.yaml new file mode 100644 index 0000000000000..5fdb6a274bd07 --- /dev/null +++ b/content/en/examples/controllers/job-success-policy-example.yaml @@ -0,0 +1,25 @@ +apiVersion: batch/v1 +kind: Job +spec: + parallelism: 10 + completions: 10 + completionMode: Indexed # Required for the feature + successPolicy: + rules: + - succeededIndexes: 0-2 + succeededCount: 1 + template: + spec: + containers: + - name: main + image: python + command: # The jobs succeed as there is one succeeded index + # among indexes 0, 1, and 2. + - python3 + - -c + - | + import os, sys + if os.environ.get("JOB_COMPLETION_INDEX") == "1": + sys.exit(0) + else: + sys.exit(1) From 105d90a04bc41f313325691ccbc845a521fa0a1d Mon Sep 17 00:00:00 2001 From: Yuki Iwai Date: Tue, 26 Mar 2024 01:50:59 +0900 Subject: [PATCH 2/7] KEP-3998: move section to before Job termination and cleanup Signed-off-by: Yuki Iwai --- .../concepts/workloads/controllers/job.md | 114 +++++++++--------- 1 file changed, 57 insertions(+), 57 deletions(-) diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index 504ce4b6f1512..6798a2661971d 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -550,6 +550,63 @@ terminating Pods only once these Pods reach the terminal `Failed` phase. This be to `podReplacementPolicy: Failed`. For more information, see [Pod replacement policy](#pod-replacement-policy). {{< /note >}} +## Success policy {#success-policy} + +{{< feature-state feature_gate_name="JobSuccessPolicy" >}} + +{{< note >}} +You can only configure a success policy for an Indexed Job if you have the +`JobSuccessPolicy` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) +enabled in your cluster. +{{< /note >}} + +When you run an indexed Job, a success policy defined with the `spec.successPolicy` field, +allows you to define when a Job can be declared as succeeded based on the number of succeeded pods. + +In some situations, you may want to have a better control when handling Pod +successes than the control provided by the `.spec.completins`. +There are some examples of use cases: + +* To optimize costs of running workloads by avoiding unnecessary Pod running, + you can terminate a Job as soon as one of its Pods succeeds. +* To care only about a leader index in determining the success or failure of a Job + in a batch workloads such as MPI and PyTorch etc. + +You can configure a success policy, in the `.spec.successPolicy` field, +to meet the above use cases. This policy can handle Job successes based on the +number of succeeded pods. After the Job meet success policy, the lingering Pods +are terminated by the Job controller. + +When you specify the only `.spec.successPolicy.rules[*].succeededIndexes`, +once all indexes specified in the `succeededIndexes` succeeded, the Job is marked as succeeded. +The `succeededIndexes` must be a list within 0 to `.spec.completions-1` and +must not contain duplicate indexes. The `succeededIndexes` is represented as intervals separated by a hyphen. +The number are listed in represented by the first and last element of the series, separated by a hyphen. +For example, if you want to specify 1, 3, 4, 5 and 7, the `succeededIndexes` is represented as `1,3-5,7`. + +When you specify the only `spec.successPolicy.rules[*].succeededCount`, +once the number of succeeded indexes reaches the `succeededCount`, the Job is marked as succeeded. + +When you specify both `succeededIndexes` and `succeededCount`, +once the number of succeeded indexes specified in the `succeededIndexes` reaches the `succeededCount`, +the Job is marked as succeeded. + +Note that when you specify multiple rules in the `.spec.succeessPolicy.rules`, +the rules are evaluated in order. Once the Job meets a rule, the remaining rules are ignored. + +Here is a manifest for a Job with `successPolicy`: + +{{% code_sample file="/controllers/job-success-policy-example.yaml" %}} + +In the example above, the rule of the success policy specifies that +the Job should be marked succeeded and terminate the lingering Pods +if one of the 0, 1, and 2 indexes succeeded. + +{{< note >}} +When you specify both a success policy and some terminating policies such as `.spec.backoffLimit` and `.spec.podFailurePolicy`, +once the Job meets both policies, the terminating policies are respected and a success policy is ignored. +{{< /note >}} + ## Job termination and cleanup When a Job completes, no more Pods are created, but the Pods are [usually](#pod-backoff-failure-policy) not deleted either. @@ -1050,63 +1107,6 @@ after the operation: the built-in Job controller and the external controller indicated by the field value. {{< /warning >}} -### Success policy {#success-policy} - -{{< feature-state for_k8s_version="v1.29" state="alpha" >}} - -{{< note >}} -You can only configure a success policy for an Indexed Job if you have the -`JobSuccessPolicy` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -enabled in your cluster. -{{< /note >}} - -When you run an indexed Job, a success policy defined with the `spec.successPolicy` field, -allows you to define when a Job can be declared as succeeded based on the number of succeeded pods. - -In some situations, you may want to have a better control when handling Pod -successes than the control provided by the `.spec.completins`. -There are some examples of use cases: - -* To optimize costs of running workloads by avoiding unnecessary Pod running, - you can terminate a Job as soon as one of its Pods succeeds. -* To care only about a leader index in determining the success or failure of a Job - in a batch workloads such as MPI and PyTorch etc. - -You can configure a success policy, in the `.spec.successPolicy` field, -to meet the above use cases. This policy can handle Job successes based on the -number of succeeded pods. After the Job meet success policy, the lingering Pods -are terminated by the Job controller. - -When you specify the only `.spec.successPolicy.rules[*].succeededIndexes`, -once all indexes specified in the `succeededIndexes` succeeded, the Job is marked as succeeded. -The `succeededIndexes` must be a list within 0 to `.spec.completions-1` and -must not contain duplicate indexes. The `succeededIndexes` is represented as intervals separated by a hyphen. -The number are listed in represented by the first and last element of the series, separated by a hyphen. -For example, if you want to specify 1, 3, 4, 5 and 7, the `succeededIndexes` is represented as `1,3-5,7`. - -When you specify the only `spec.successPolicy.rules[*].succeededCount`, -once the number of succeeded indexes reaches the `succeededCount`, the Job is marked as succeeded. - -When you specify both `succeededIndexes` and `succeededCount`, -once the number of succeeded indexes specified in the `succeededIndexes` reaches the `succeededCount`, -the Job is marked as succeeded. - -Note that when you specify multiple rules in the `.spec.succeessPolicy.rules`, -the rules are evaluated in order. Once the Job meets a rule, the remaining rules are ignored. - -Here is a manifest for a Job with `successPolicy`: - -{{% code_sample file="/controllers/job-success-policy-example.yaml" %}} - -In the example above, the rule of the success policy specifies that -the Job should be marked succeeded and terminate the lingering Pods -if one of the 0, 1, and 2 indexes succeeded. - -{{< note >}} -When you specify both a success policy and some terminating policies such as `.spec.backoffLimit` and `.spec.podFailurePolicy`, -once the Job meets both policies, the terminating policies are respected and a success policy is ignored. -{{< /note >}} - ## Alternatives ### Bare Pods From 32fd60c16ed207e32fe5c59757f3fd80fe35729c Mon Sep 17 00:00:00 2001 From: Yuki Iwai Date: Tue, 26 Mar 2024 02:09:25 +0900 Subject: [PATCH 3/7] KEP-3998: Make the explanation for rules more clarify Signed-off-by: Yuki Iwai --- .../docs/concepts/workloads/controllers/job.md | 18 ++++++++---------- .../job-success-policy-example.yaml | 6 +++--- 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index 6798a2661971d..a7845027d7d94 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -577,17 +577,12 @@ to meet the above use cases. This policy can handle Job successes based on the number of succeeded pods. After the Job meet success policy, the lingering Pods are terminated by the Job controller. -When you specify the only `.spec.successPolicy.rules[*].succeededIndexes`, +* When you specify the only `.spec.successPolicy.rules[*].succeededIndexes`, once all indexes specified in the `succeededIndexes` succeeded, the Job is marked as succeeded. -The `succeededIndexes` must be a list within 0 to `.spec.completions-1` and -must not contain duplicate indexes. The `succeededIndexes` is represented as intervals separated by a hyphen. -The number are listed in represented by the first and last element of the series, separated by a hyphen. -For example, if you want to specify 1, 3, 4, 5 and 7, the `succeededIndexes` is represented as `1,3-5,7`. - -When you specify the only `spec.successPolicy.rules[*].succeededCount`, +The `succeededIndexes` must be a list of intervals between 0 and `.spec.completions-1`. +* When you specify the only `spec.successPolicy.rules[*].succeededCount`, once the number of succeeded indexes reaches the `succeededCount`, the Job is marked as succeeded. - -When you specify both `succeededIndexes` and `succeededCount`, +* When you specify both `succeededIndexes` and `succeededCount`, once the number of succeeded indexes specified in the `succeededIndexes` reaches the `succeededCount`, the Job is marked as succeeded. @@ -600,7 +595,10 @@ Here is a manifest for a Job with `successPolicy`: In the example above, the rule of the success policy specifies that the Job should be marked succeeded and terminate the lingering Pods -if one of the 0, 1, and 2 indexes succeeded. +if one of the 0, 2, and 3 indexes succeeded. + +Note that the `succeededIndexes` is represented as intervals separated by a hyphen. +The number are listed in represented by the first and last element of the series, separated by a hyphen. {{< note >}} When you specify both a success policy and some terminating policies such as `.spec.backoffLimit` and `.spec.podFailurePolicy`, diff --git a/content/en/examples/controllers/job-success-policy-example.yaml b/content/en/examples/controllers/job-success-policy-example.yaml index 5fdb6a274bd07..f82115fe2842f 100644 --- a/content/en/examples/controllers/job-success-policy-example.yaml +++ b/content/en/examples/controllers/job-success-policy-example.yaml @@ -6,7 +6,7 @@ spec: completionMode: Indexed # Required for the feature successPolicy: rules: - - succeededIndexes: 0-2 + - succeededIndexes: 0,2-3 succeededCount: 1 template: spec: @@ -14,12 +14,12 @@ spec: - name: main image: python command: # The jobs succeed as there is one succeeded index - # among indexes 0, 1, and 2. + # among indexes 0, 2, and 3. - python3 - -c - | import os, sys - if os.environ.get("JOB_COMPLETION_INDEX") == "1": + if os.environ.get("JOB_COMPLETION_INDEX") == "2": sys.exit(0) else: sys.exit(1) From cec3c3f38802c5438be3fd7cb149b66eb038a513 Mon Sep 17 00:00:00 2001 From: Yuki Iwai Date: Tue, 26 Mar 2024 02:22:42 +0900 Subject: [PATCH 4/7] KEP-3998: Apply Aldo's suggestons Signed-off-by: Yuki Iwai --- .../concepts/workloads/controllers/job.md | 39 +++++++++---------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index a7845027d7d94..984122dfa0dc1 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -560,31 +560,30 @@ You can only configure a success policy for an Indexed Job if you have the enabled in your cluster. {{< /note >}} -When you run an indexed Job, a success policy defined with the `spec.successPolicy` field, -allows you to define when a Job can be declared as succeeded based on the number of succeeded pods. +When creating an Indexed Job, you can define when a Job can be declared as succeeded using a `.spec.successPolicy`, +based on the pods that succeeded. -In some situations, you may want to have a better control when handling Pod -successes than the control provided by the `.spec.completins`. -There are some examples of use cases: +By default, a Job succeeds when the number of succeeded Pods equals `.spec.completions`. +These are some situations where you might want additional control for declaring a Job succeeded: -* To optimize costs of running workloads by avoiding unnecessary Pod running, - you can terminate a Job as soon as one of its Pods succeeds. -* To care only about a leader index in determining the success or failure of a Job - in a batch workloads such as MPI and PyTorch etc. +* When running simulations with different parameters, + you might not need all the simulations to succeed for the overall Job to be successful. +* When following a leader-worker pattern, only the success of the leader determines the success or + failure of a Job. Examples of this are frameworks like MPI and PyTorch etc. You can configure a success policy, in the `.spec.successPolicy` field, to meet the above use cases. This policy can handle Job successes based on the -number of succeeded pods. After the Job meet success policy, the lingering Pods -are terminated by the Job controller. - -* When you specify the only `.spec.successPolicy.rules[*].succeededIndexes`, -once all indexes specified in the `succeededIndexes` succeeded, the Job is marked as succeeded. -The `succeededIndexes` must be a list of intervals between 0 and `.spec.completions-1`. -* When you specify the only `spec.successPolicy.rules[*].succeededCount`, -once the number of succeeded indexes reaches the `succeededCount`, the Job is marked as succeeded. +succeeded pods. After the Job meet success policy, the job controller terminates the lingering Pods. +A success policy is defined by rules. Each rule can take one of the following forms: + +* When you specify the `succeededIndexes` only, + once all indexes specified in the `succeededIndexes` succeeded, the Job is marked as succeeded. + The `succeededIndexes` must be a list of intervals between 0 and `.spec.completions-1`. +* When you specify the `succeededCount` only, + once the number of succeeded indexes reaches the `succeededCount`, the Job is marked as succeeded. * When you specify both `succeededIndexes` and `succeededCount`, -once the number of succeeded indexes specified in the `succeededIndexes` reaches the `succeededCount`, -the Job is marked as succeeded. + once the number of succeeded indexes specified in the `succeededIndexes` reaches the `succeededCount`, + the Job is marked as succeeded. Note that when you specify multiple rules in the `.spec.succeessPolicy.rules`, the rules are evaluated in order. Once the Job meets a rule, the remaining rules are ignored. @@ -602,7 +601,7 @@ The number are listed in represented by the first and last element of the series {{< note >}} When you specify both a success policy and some terminating policies such as `.spec.backoffLimit` and `.spec.podFailurePolicy`, -once the Job meets both policies, the terminating policies are respected and a success policy is ignored. +once the Job meets either policy, the job controller respects the terminating policy and ignores the success policy. {{< /note >}} ## Job termination and cleanup From fcdb477aa486e7e5d594ea8a7c1ff31a0a2e1c1e Mon Sep 17 00:00:00 2001 From: Yuki Iwai Date: Tue, 26 Mar 2024 02:34:20 +0900 Subject: [PATCH 5/7] KEP-3998: Mention SuccessCriteriaMet condition Signed-off-by: Yuki Iwai --- content/en/docs/concepts/workloads/controllers/job.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index 984122dfa0dc1..4175329eaeba0 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -595,6 +595,8 @@ Here is a manifest for a Job with `successPolicy`: In the example above, the rule of the success policy specifies that the Job should be marked succeeded and terminate the lingering Pods if one of the 0, 2, and 3 indexes succeeded. +The Job that met the success policy gets the `SuccessCriteriaMet` condition. +After the removal of the lingering Pods is issued, the Job gets the `Complete` condition. Note that the `succeededIndexes` is represented as intervals separated by a hyphen. The number are listed in represented by the first and last element of the series, separated by a hyphen. From d79de0290a84b448ca4aa61b6ebc8cb050161dba Mon Sep 17 00:00:00 2001 From: Yuki Iwai Date: Tue, 26 Mar 2024 02:41:36 +0900 Subject: [PATCH 6/7] KEP-3998: Clarify the example Signed-off-by: Yuki Iwai --- content/en/docs/concepts/workloads/controllers/job.md | 2 +- ...-success-policy-example.yaml => job-success-policy.yaml} | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) rename content/en/examples/controllers/{job-success-policy-example.yaml => job-success-policy.yaml} (66%) diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index 4175329eaeba0..cfb9a0c0bd178 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -590,7 +590,7 @@ the rules are evaluated in order. Once the Job meets a rule, the remaining rules Here is a manifest for a Job with `successPolicy`: -{{% code_sample file="/controllers/job-success-policy-example.yaml" %}} +{{% code_sample file="/controllers/job-success-policy.yaml" %}} In the example above, the rule of the success policy specifies that the Job should be marked succeeded and terminate the lingering Pods diff --git a/content/en/examples/controllers/job-success-policy-example.yaml b/content/en/examples/controllers/job-success-policy.yaml similarity index 66% rename from content/en/examples/controllers/job-success-policy-example.yaml rename to content/en/examples/controllers/job-success-policy.yaml index f82115fe2842f..1f7927b2f34fc 100644 --- a/content/en/examples/controllers/job-success-policy-example.yaml +++ b/content/en/examples/controllers/job-success-policy.yaml @@ -3,7 +3,7 @@ kind: Job spec: parallelism: 10 completions: 10 - completionMode: Indexed # Required for the feature + completionMode: Indexed # Required for the success policy successPolicy: rules: - succeededIndexes: 0,2-3 @@ -13,8 +13,8 @@ spec: containers: - name: main image: python - command: # The jobs succeed as there is one succeeded index - # among indexes 0, 2, and 3. + command: # Provided that at least one of the Pods with 0, 2, and 3 indexes has succeeded, + # the overall Job is a success. - python3 - -c - | From 74652566e3d50a7775a96284ba19b4ef1bab9eda Mon Sep 17 00:00:00 2001 From: Yuki Iwai Date: Tue, 26 Mar 2024 02:49:43 +0900 Subject: [PATCH 7/7] KEP-3998: Aldo's second suggestions Signed-off-by: Yuki Iwai --- .../en/docs/concepts/workloads/controllers/job.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index cfb9a0c0bd178..1701af7f79550 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -572,21 +572,21 @@ These are some situations where you might want additional control for declaring failure of a Job. Examples of this are frameworks like MPI and PyTorch etc. You can configure a success policy, in the `.spec.successPolicy` field, -to meet the above use cases. This policy can handle Job successes based on the +to meet the above use cases. This policy can handle Job success based on the succeeded pods. After the Job meet success policy, the job controller terminates the lingering Pods. A success policy is defined by rules. Each rule can take one of the following forms: * When you specify the `succeededIndexes` only, - once all indexes specified in the `succeededIndexes` succeeded, the Job is marked as succeeded. + once all indexes specified in the `succeededIndexes` succeed, the job controller marks the Job as succeeded. The `succeededIndexes` must be a list of intervals between 0 and `.spec.completions-1`. * When you specify the `succeededCount` only, - once the number of succeeded indexes reaches the `succeededCount`, the Job is marked as succeeded. + once the number of succeeded indexes reaches the `succeededCount`, the job controller marks the Job as succeeded. * When you specify both `succeededIndexes` and `succeededCount`, - once the number of succeeded indexes specified in the `succeededIndexes` reaches the `succeededCount`, - the Job is marked as succeeded. + once the number of succeeded indexes from the subset of indexes specified in the `succeededIndexes` reaches the `succeededCount`, + the job controller marks the Job as succeeded. Note that when you specify multiple rules in the `.spec.succeessPolicy.rules`, -the rules are evaluated in order. Once the Job meets a rule, the remaining rules are ignored. +the job controller evaluates the rules in order. Once the Job meets a rule, the job controller ignores remaining rules. Here is a manifest for a Job with `successPolicy`: