Support for the Job managedBy field (alpha)

[mimowo]

What type of PR is this?

/kind feature
/kind api-change

What this PR does / why we need it:

In order to support a mechanism allowing to delegate reconciliation of a Job to an external controller. See more details in the KEP.

Which issue(s) this PR fixes:

Part of enhancement tracking issue: kubernetes/enhancements#4368

Special notes for your reviewer:

After discussion with API reviewers we decided to use field instead of label, see here.

This means the feature will go into Alpha for 1.30. Here is the relevant KEP update.

Does this PR introduce a user-facing change?

Added (alpha) support for the managedBy field on Jobs. Jobs with a custom value of this field - any
value other than `kubernetes.io/job-controller` - are skipped by the job controller, and their
reconciliation is delegated to an external controller, indicated by the value of the field. Jobs that
don't have this field at all, or where the field value is the reserved string `kubernetes.io/job-controller`,
are reconciled by the built-in job controller.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

https://github.com/kubernetes/enhancements/tree/master/keps/sig-apps/4368-support-managed-by-for-batch-jobs

[mimowo]

/cc @alculquicondor

[sftim]

Changelog suggestion

-Add support for the Job managed-by label in Beta
+Added more labelling for Pods that belong to a Job. The Pods are labelled with the key `batch.kubernetes.io/managed-by`
+provided that the feature gate `JobManagedByLabel` is enabled for the cluster. The label value is `job-controller.k8s.io`
+for Pods managed by a Job.

(if that's not right, please correct the text to tell the actual story)

[mimowo]

Changelog suggestion

-Add support for the Job managed-by label in Beta
+Added more labelling for Pods that belong to a Job. The Pods are labelled with the key `batch.kubernetes.io/managed-by`
+provided that the feature gate `JobManagedByLabel` is enabled for the cluster. The label value is `job-controller.k8s.io`
+for Pods managed by a Job.

(if that's not right, please correct the text to tell the actual story)

Thanks for pointing out there is a need to clarify this. Let me know if the new note makes it clear, I'm happy to adjust / clarify more.

[sftim]

Further changelog suggestion

-Add support for the Job's "batch.kubernetes.io/managed-by" label in Beta. Jobs with a custom value of this
-label (other than "job-controller.k8s.io") are skipped by the Job controller, and their reconciliation is
+Added (beta) support for the `batch.kubernetes.io/managed-by` label on Jobs. Jobs with a custom value of this
+label - any label value other than `job-controller.k8s.io` - are skipped by the job controller, and their reconciliation is
-delegated to an external controller, indicated by the value of the label. Jobs without this label, or with 
-the reserved "job-controller.k8s.io" as value, are reconciled by the built-in Job controller.
+delegated to an external controller, indicated by the value of the label. Jobs that don't have this label at all,
+or where the label value is the reserved string `job-controller.k8s.io` as value, are reconciled by the built-in
+job controller.

This is [a fragment of] Markdown, so use backticks rather than double quotes. Also no capitals in “job controller”.

[mimowo]

Further changelog suggestion

lgtm, thanks, applied

[sftim]

D'oh, my error; one more suggested update.

 Added (beta) support for the `batch.kubernetes.io/managed-by` label on Jobs. Jobs with a custom value of this
 label - any label value other than `job-controller.k8s.io` - are skipped by the job controller, and their reconciliation is
 delegated to an external controller, indicated by the value of the label. Jobs that don't have this label at all,
-or where the label value is the reserved string `job-controller.k8s.io` as value, are reconciled by the built-in
+or where the label value is the reserved string `job-controller.k8s.io`, are reconciled by the built-in
 job controller.

[mimowo]

The failure in the TestNonParallelJob integration test looks like unrelated flake, and I got this failure locally on master as well. I will investigate.

EDIT: I see what is going on, fix: #123321. I saw it failing locally because I was looping the test.

[mimowo]

/retest

[alculquicondor]

/approve cancel
to skip updating the pod metrics for finalized pods.

[mimowo]

/approve cancel
to skip updating the pod metrics for finalized pods.

Same response as here: #123273 (comment). I'm afraid that making the distinction is likely to introduce bugs. For example, the metic is guage. We may not increment because there is a custom managedBy, but then we would decrement the metric on pod removal (becuase there is already no parent, so we cannot check). This could make the metric skewed, like getting too early to 0.

[mimowo]

/retest

[alculquicondor]

/approve
/hold
for soltysh

[mimowo]

/retest

[soltysh]

/lgtm
/label tide/merge-method-squash

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: a1e9e235b583a9b058118835617e84fd5b6d06d4

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alculquicondor, deads2k, mimowo, soltysh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• api/OWNERS [deads2k]
• pkg/apis/OWNERS [deads2k]
• pkg/controller/job/OWNERS [alculquicondor,deads2k,soltysh]
• pkg/features/OWNERS [deads2k,soltysh]
• pkg/generated/openapi/OWNERS [deads2k]
• pkg/registry/OWNERS [deads2k]
• staging/src/k8s.io/api/OWNERS [deads2k]
• staging/src/k8s.io/client-go/applyconfigurations/OWNERS [deads2k]
• test/integration/job/OWNERS [alculquicondor,deads2k,soltysh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[soltysh]

/hold cancel

[mimowo]

/test pull-kubernetes-e2e-kind-ipv6
unrelated failure