Use cert-manager to manage certificates and enhance RBAC security #1978
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: ChenYi015 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
ChenYi015
force-pushed
the
improvement/rbac
branch
from
f79a737
to
abf9349
Compare
ChenYi015
force-pushed
the
improvement/rbac
branch
2 times, most recently
from
ec02dca
to
2e225b8
Compare
ChenYi015
changed the title
Signed-off-by: Yi Chen <github@chenyicn.net>
ChenYi015
force-pushed
the
improvement/rbac
branch
from
2e225b8
to
a92b2e6
Compare
|
@ChenYi015, thanks for the enhancements to RBAC and the webhook; they look good. Since these modifications are breaking changes, we need to ensure they are thoroughly tested across various scenarios that utilize the webhook configuration. Could you please provide evidence of Spark job execution to demonstrate their effectiveness? FYI @yuchaoran2011 |
Signed-off-by: Yi Chen <github@chenyicn.net>
|
I had did some e2e tests manually as follows:
# kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
kind create cluster --config kind-config.yaml
docker build -t docker.io/kubeflow/spark-operator:local .
kind load docker-image docker.io/kubeflow/spark-operator:local
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml
$ helm install spark-operator charts/spark-operator-chart \
--namespace spark-operator \
--create-namespace \
--set image.tag=local \
--set webhook.enable=true \
--set enforceQuotaEnforcement=true \
--debug
apiVersion: v1
data:
ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURrakNDQW5xZ0F3SUJBZ0lSQUozQVpMTXRpQ1NSTUVOaGtleXFDNXd3RFFZSktvWklodmNOQVFFTEJRQXcKSlRFak1DRUdBMVVFQXhNYWMzQmhjbXN0YjNCbGNtRjBiM0l0ZDJWaWFHOXZheTF6ZG1Nd0lCY05NalF3TkRFNQpNRGd4T1RVeVdoZ1BNakk1T0RBeU1ESXdPREU1TlRKYU1DVXhJekFoQmdOVkJBTVRHbk53WVhKckxXOXdaWEpoCmRHOXlMWGRsWW1odmIyc3RjM1pqTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUEKcHJiakNpcEFHRjl4ejl6YWRNalFTWGhsanZVbmVpdjRYck05dUl0dFVsNVN3bHZpRjNHaEJUQ283cjY2M2IrQwpxTlAwWS82RTlXMTF3YURzcExleGI5REhsZG1SSjBYaWpGbkhHQmI0aUpHZlVZdGRkU2lwVDZDQlR6eitBM3lQCmhuOVU0TWlmMGZ0VlgzbTUraHBleTJvaGVWc1FQVHdKOU8vQ3YyeEFwbzR1cGJaVzNFeEhiNFh3WGc1Y3pTaHYKV1RNYWhjbk53SFgrSnBpajlGRGIvK3BUQ3dJKzlxdTNsNWd3eXhXb28yb282cmtXK0g4OXZKdjJrSHBud2FYLwpkUU1KWENKQWtBKzVhVHMxZjc3WWE5SG8xMkZOUFJSZjBpTWk2S2lTYlZMeTZCWHd3TXV0akFKSHZrczJXNXI5ClAxZVAzNk0vK1ovUVI1a0ltNW5oelFJREFRQUJvNEc2TUlHM01BNEdBMVVkRHdFQi93UUVBd0lDcERBUEJnTlYKSFJNQkFmOEVCVEFEQVFIL01CMEdBMVVkRGdRV0JCUmgrR3RSOEtnQkY2NithZ1VtZVhyOVNmdVNYVEIxQmdOVgpIUkVFYmpCc2dpMXpjR0Z5YXkxdmNHVnlZWFJ2Y2kxM1pXSm9iMjlyTFhOMll5NXpjR0Z5YXkxdmNHVnlZWFJ2CmNpNXpkbU9DTzNOd1lYSnJMVzl3WlhKaGRHOXlMWGRsWW1odmIyc3RjM1pqTG5Od1lYSnJMVzl3WlhKaGRHOXkKTG5OMll5NWpiSFZ6ZEdWeUxteHZZMkZzTUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCQVFCWnhTcjl3ZjE5WG1hUwpTSEt2TjVDemdDN2c4YzEvdk9GRWZ3bld3YldJMUtRYlN1N2M4cTRqZ2xoOWhGa0ljN0JuSFRTSWF1cU5lZUZqCnFGNEw0a2dTSGk5dWJXRlBwZ0xoRnorM2dBa1BNcWQzNi8wb2d2RGYxRXpXajJVS1AzTkJrVmtoN05rd2NXcEQKVXowSUNYL3JEWFI1TmxoKzNLT1FDV3NZU25sSmZSZnBwbFRMYURzNTZlQytwM3B5OUFsRW9KSFpWSy9pMzJqbApTSXZveW5jNktPa1IyVmx3L0ViekwwdGU2dGVESVpsR2M5ZkVEY3gxSUMvSDhHems0UVZoMWxTOHB4enkrT3F0CkdCTGM3alhlVER6MjA4bmdvYzN2WlExd2xFUE5NY0M0Z3AweWdPYjRURjZXbDFsblFJTXFrZ2IwVGZKVmUrWXcKRWNkMmYzeisKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURrakNDQW5xZ0F3SUJBZ0lSQUozQVpMTXRpQ1NSTUVOaGtleXFDNXd3RFFZSktvWklodmNOQVFFTEJRQXcKSlRFak1DRUdBMVVFQXhNYWMzQmhjbXN0YjNCbGNtRjBiM0l0ZDJWaWFHOXZheTF6ZG1Nd0lCY05NalF3TkRFNQpNRGd4T1RVeVdoZ1BNakk1T0RBeU1ESXdPREU1TlRKYU1DVXhJekFoQmdOVkJBTVRHbk53WVhKckxXOXdaWEpoCmRHOXlMWGRsWW1odmIyc3RjM1pqTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUEKcHJiakNpcEFHRjl4ejl6YWRNalFTWGhsanZVbmVpdjRYck05dUl0dFVsNVN3bHZpRjNHaEJUQ283cjY2M2IrQwpxTlAwWS82RTlXMTF3YURzcExleGI5REhsZG1SSjBYaWpGbkhHQmI0aUpHZlVZdGRkU2lwVDZDQlR6eitBM3lQCmhuOVU0TWlmMGZ0VlgzbTUraHBleTJvaGVWc1FQVHdKOU8vQ3YyeEFwbzR1cGJaVzNFeEhiNFh3WGc1Y3pTaHYKV1RNYWhjbk53SFgrSnBpajlGRGIvK3BUQ3dJKzlxdTNsNWd3eXhXb28yb282cmtXK0g4OXZKdjJrSHBud2FYLwpkUU1KWENKQWtBKzVhVHMxZjc3WWE5SG8xMkZOUFJSZjBpTWk2S2lTYlZMeTZCWHd3TXV0akFKSHZrczJXNXI5ClAxZVAzNk0vK1ovUVI1a0ltNW5oelFJREFRQUJvNEc2TUlHM01BNEdBMVVkRHdFQi93UUVBd0lDcERBUEJnTlYKSFJNQkFmOEVCVEFEQVFIL01CMEdBMVVkRGdRV0JCUmgrR3RSOEtnQkY2NithZ1VtZVhyOVNmdVNYVEIxQmdOVgpIUkVFYmpCc2dpMXpjR0Z5YXkxdmNHVnlZWFJ2Y2kxM1pXSm9iMjlyTFhOMll5NXpjR0Z5YXkxdmNHVnlZWFJ2CmNpNXpkbU9DTzNOd1lYSnJMVzl3WlhKaGRHOXlMWGRsWW1odmIyc3RjM1pqTG5Od1lYSnJMVzl3WlhKaGRHOXkKTG5OMll5NWpiSFZ6ZEdWeUxteHZZMkZzTUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCQVFCWnhTcjl3ZjE5WG1hUwpTSEt2TjVDemdDN2c4YzEvdk9GRWZ3bld3YldJMUtRYlN1N2M4cTRqZ2xoOWhGa0ljN0JuSFRTSWF1cU5lZUZqCnFGNEw0a2dTSGk5dWJXRlBwZ0xoRnorM2dBa1BNcWQzNi8wb2d2RGYxRXpXajJVS1AzTkJrVmtoN05rd2NXcEQKVXowSUNYL3JEWFI1TmxoKzNLT1FDV3NZU25sSmZSZnBwbFRMYURzNTZlQytwM3B5OUFsRW9KSFpWSy9pMzJqbApTSXZveW5jNktPa1IyVmx3L0ViekwwdGU2dGVESVpsR2M5ZkVEY3gxSUMvSDhHems0UVZoMWxTOHB4enkrT3F0CkdCTGM3alhlVER6MjA4bmdvYzN2WlExd2xFUE5NY0M0Z3AweWdPYjRURjZXbDFsblFJTXFrZ2IwVGZKVmUrWXcKRWNkMmYzeisKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBcHJiakNpcEFHRjl4ejl6YWRNalFTWGhsanZVbmVpdjRYck05dUl0dFVsNVN3bHZpCkYzR2hCVENvN3I2NjNiK0NxTlAwWS82RTlXMTF3YURzcExleGI5REhsZG1SSjBYaWpGbkhHQmI0aUpHZlVZdGQKZFNpcFQ2Q0JUenorQTN5UGhuOVU0TWlmMGZ0VlgzbTUraHBleTJvaGVWc1FQVHdKOU8vQ3YyeEFwbzR1cGJaVwozRXhIYjRYd1hnNWN6U2h2V1RNYWhjbk53SFgrSnBpajlGRGIvK3BUQ3dJKzlxdTNsNWd3eXhXb28yb282cmtXCitIODl2SnYya0hwbndhWC9kUU1KWENKQWtBKzVhVHMxZjc3WWE5SG8xMkZOUFJSZjBpTWk2S2lTYlZMeTZCWHcKd011dGpBSkh2a3MyVzVyOVAxZVAzNk0vK1ovUVI1a0ltNW5oelFJREFRQUJBb0lCQURQb3dpektGd2Ivd1BwTgo5QjlrMzZYZ3ZRNlVPdVNKaUY1RVJUV1BCRTBmWFhsekRxVHFGbzc5SmJ5ZzJxSnRIUXZnWXpvM1o4TGJnRXo0CkJIY05VMHhUbjdHQyt2VlZoMExZT1d2QXkwWUt3YWFzVkQ5RWwrTlpRS29oZlo0c1RLdytQWGw0UG5zdFhObkwKbHlMT3p6RmJMRzBTODYxOVlrUkgyMFdyT1Q0aWlvbzN0V1ByOHpLSmxPWDhWbnRZcjlXTGVIZ1Fvem90UGhxeAovODdyejFXWHFrRnpCc2dFOTZ1REZMaVBXNkY3NEc2T3h2YkVWWFdGTWQ0ZEpVRFJzTnVLNzJodHdMR1FUeFpGCld4Y3UxcGtOL3F6TU44K01FaWZOMjBZVkFIb090UUxDUWlWbzUvQmtWOFhvQ0dvdWxJSGtZVktnMDFzbGRsTUgKV1plMWh1a0NnWUVBMk54c0toS0VSMVc4ZDhQc25SU1ZHVWMvSGF0OWlHVkdjSFdFeXI4eVlMNGF4VHUyTjVIbQpjTlZkN3NrOHdoWEROTmUvR0MvNUxyMnhsTWZ0SEFPMDlBSEI4c1k2VHBkY2RJM3VDQmVKdjBDS3ZWV050dHVOCndSd3owaEFxVVk4bmt4OGREUlk0MGpDb0RJQytCRDlMUTczTDZqMDd2YXdacXVzMVNJM1hIUE1DZ1lFQXhNMk4KVW8zbDYyVVRCTmJWOGJTZkk1d2kxUEIyMVdRMlpsTkRJMWErenIyKy92clFMYkdteTQ2N2EvQXV0VkNjM2piMwphcnFoMnk5MU1QREkzbmo3Q096NHZXdmRWOFhhb01nVGs4VXJVTkpNa1YyQ1ZoT1lnMm5paC9oNjlhSDh0NXZyCkowUERsMUhkV21aZ1lUcG9ZVTV6Sks1TG9pc1c0Q0V1QmZmWHRqOENnWUVBaHR4dEY2ejBQamgvUUN2RGxkd2EKN09GMXlzbS9BM2VYMm8vNnlPc21OU3VFRW13VDRYTUdUOTBnMFVLc1dqYVdjdUpJb0k0VTlKQmJReDhod1c0OAp0aEh4dnp5dlBmaTdNbkNoT3I4SWxydFlqRzRMOTArS1ZERnB1SlQ4L0VOK1pPdlFjVGNFbHhKVEw1TlFsK3VVCnJzZjFhL0l3bUwvU0pveW5VanVETHlrQ2dZQXhseUlmUlVaRVZJbkFWR3BBcDZUYUVrUHNxRk1wWGtFZTg2Z2QKUllSN0QvRUt5eVNkL3YvcWx5Sy9CaHllNzVzM20xNG85TEpJU3VNYTIvR2hyWmFFSDBKbWFNR3psbFdLNGgrYwp3cmd4ZTJ4enpMNUU5bGNkOWhadzhMbjdIMEJRKytiN1UrWDdRVk9ucmRoblo1Slp3cXpjOTFaem5LdjgraDlnCnZaMExmUUtCZ0RQcnJSekk2aHhZV2piM2tudUIxNE53bitYa0RnOHEyaHV0ZThFeXZGUStRaFVzRS9GSlNqbjMKbHNzNWRJUkVyaiticTFvOUxtTTgrRnhUQUUzcXJaVyt4MWxvVFNZUDZ3d2ZSTVNDVTZxeFVQbWJRdTN6SnhiQwpUQmZvTjJFZCtDRnJoelRWY3ZZRTlHcUkrWlpjcGVTbkRmTjZPN0tGWjZXUURUYnB3TjdkCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
kind: Secret
metadata:
annotations:
cert-manager.io/allow-direct-injection: "true"
cert-manager.io/alt-names: spark-operator-webhook-svc.spark-operator.svc,spark-operator-webhook-svc.spark-operator.svc.cluster.local
cert-manager.io/certificate-name: spark-operator-cert
cert-manager.io/common-name: spark-operator-webhook-svc
cert-manager.io/ip-sans: ""
cert-manager.io/issuer-group: ""
cert-manager.io/issuer-kind: Issuer
cert-manager.io/issuer-name: spark-operator-issuer
cert-manager.io/uri-sans: ""
meta.helm.sh/release-name: spark-operator
meta.helm.sh/release-namespace: spark-operator
creationTimestamp: "2024-04-19T08:19:52Z"
labels:
app.kubernetes.io/managed-by: Helm
controller.cert-manager.io/fao: "true"
name: spark-operator-webhook-tls
namespace: spark-operator
resourceVersion: "2532"
uid: 4db42d4b-8292-415a-afac-a0c2b5263351
type: Opaque
$ kubectl get mutatingwebhookconfigurations.admissionregistration.k8s.io spark-operator-webhook -o yaml
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
annotations:
cert-manager.io/inject-ca-from-secret: spark-operator/spark-operator-webhook-tls
meta.helm.sh/release-name: spark-operator
meta.helm.sh/release-namespace: spark-operator
creationTimestamp: "2024-04-19T08:19:52Z"
generation: 3
labels:
app.kubernetes.io/instance: spark-operator
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: spark-operator
app.kubernetes.io/version: v1beta2-1.5.0-3.5.0
helm.sh/chart: spark-operator-1.3.0
name: spark-operator-webhook
resourceVersion: "2533"
uid: a853d687-5723-40dc-ab6a-a540eece685d
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURrakNDQW5xZ0F3SUJBZ0lSQUozQVpMTXRpQ1NSTUVOaGtleXFDNXd3RFFZSktvWklodmNOQVFFTEJRQXcKSlRFak1DRUdBMVVFQXhNYWMzQmhjbXN0YjNCbGNtRjBiM0l0ZDJWaWFHOXZheTF6ZG1Nd0lCY05NalF3TkRFNQpNRGd4T1RVeVdoZ1BNakk1T0RBeU1ESXdPREU1TlRKYU1DVXhJekFoQmdOVkJBTVRHbk53WVhKckxXOXdaWEpoCmRHOXlMWGRsWW1odmIyc3RjM1pqTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUEKcHJiakNpcEFHRjl4ejl6YWRNalFTWGhsanZVbmVpdjRYck05dUl0dFVsNVN3bHZpRjNHaEJUQ283cjY2M2IrQwpxTlAwWS82RTlXMTF3YURzcExleGI5REhsZG1SSjBYaWpGbkhHQmI0aUpHZlVZdGRkU2lwVDZDQlR6eitBM3lQCmhuOVU0TWlmMGZ0VlgzbTUraHBleTJvaGVWc1FQVHdKOU8vQ3YyeEFwbzR1cGJaVzNFeEhiNFh3WGc1Y3pTaHYKV1RNYWhjbk53SFgrSnBpajlGRGIvK3BUQ3dJKzlxdTNsNWd3eXhXb28yb282cmtXK0g4OXZKdjJrSHBud2FYLwpkUU1KWENKQWtBKzVhVHMxZjc3WWE5SG8xMkZOUFJSZjBpTWk2S2lTYlZMeTZCWHd3TXV0akFKSHZrczJXNXI5ClAxZVAzNk0vK1ovUVI1a0ltNW5oelFJREFRQUJvNEc2TUlHM01BNEdBMVVkRHdFQi93UUVBd0lDcERBUEJnTlYKSFJNQkFmOEVCVEFEQVFIL01CMEdBMVVkRGdRV0JCUmgrR3RSOEtnQkY2NithZ1VtZVhyOVNmdVNYVEIxQmdOVgpIUkVFYmpCc2dpMXpjR0Z5YXkxdmNHVnlZWFJ2Y2kxM1pXSm9iMjlyTFhOMll5NXpjR0Z5YXkxdmNHVnlZWFJ2CmNpNXpkbU9DTzNOd1lYSnJMVzl3WlhKaGRHOXlMWGRsWW1odmIyc3RjM1pqTG5Od1lYSnJMVzl3WlhKaGRHOXkKTG5OMll5NWpiSFZ6ZEdWeUxteHZZMkZzTUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCQVFCWnhTcjl3ZjE5WG1hUwpTSEt2TjVDemdDN2c4YzEvdk9GRWZ3bld3YldJMUtRYlN1N2M4cTRqZ2xoOWhGa0ljN0JuSFRTSWF1cU5lZUZqCnFGNEw0a2dTSGk5dWJXRlBwZ0xoRnorM2dBa1BNcWQzNi8wb2d2RGYxRXpXajJVS1AzTkJrVmtoN05rd2NXcEQKVXowSUNYL3JEWFI1TmxoKzNLT1FDV3NZU25sSmZSZnBwbFRMYURzNTZlQytwM3B5OUFsRW9KSFpWSy9pMzJqbApTSXZveW5jNktPa1IyVmx3L0ViekwwdGU2dGVESVpsR2M5ZkVEY3gxSUMvSDhHems0UVZoMWxTOHB4enkrT3F0CkdCTGM3alhlVER6MjA4bmdvYzN2WlExd2xFUE5NY0M0Z3AweWdPYjRURjZXbDFsblFJTXFrZ2IwVGZKVmUrWXcKRWNkMmYzeisKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
service:
name: spark-operator-webhook-svc
namespace: spark-operator
path: /webhook
port: 8080
failurePolicy: Ignore
matchPolicy: Equivalent
name: webhook.sparkoperator.k8s.io
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- default
objectSelector:
matchExpressions:
- key: sparkoperator.k8s.io/app-name
operator: Exists
- key: spark-role
operator: Exists
matchLabels:
sparkoperator.k8s.io/launched-by-spark-operator: "true"
reinvocationPolicy: Never
rules:
- apiGroups:
- ""
apiVersions:
- v1
operations:
- CREATE
resources:
- pods
scope: '*'
sideEffects: NoneOnDryRun
timeoutSeconds: 30
$ kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io -n spark-operator -o yaml
apiVersion: v1
items:
- apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
annotations:
cert-manager.io/inject-ca-from-secret: cert-manager/cert-manager-webhook-ca
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"admissionregistration.k8s.io/v1","kind":"ValidatingWebhookConfiguration","metadata":{"annotations":{"cert-manager.io/inject-ca-from-secret":"cert-manager/cert-manager-webhook-ca"},"labels":{"app":"webhook","app.kubernetes.io/component":"webhook","app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/name":"webhook","app.kubernetes.io/version":"v1.14.4"},"name":"cert-manager-webhook"},"webhooks":[{"admissionReviewVersions":["v1"],"clientConfig":{"service":{"name":"cert-manager-webhook","namespace":"cert-manager","path":"/validate"}},"failurePolicy":"Fail","matchPolicy":"Equivalent","name":"webhook.cert-manager.io","namespaceSelector":{"matchExpressions":[{"key":"cert-manager.io/disable-validation","operator":"NotIn","values":["true"]}]},"rules":[{"apiGroups":["cert-manager.io","acme.cert-manager.io"],"apiVersions":["v1"],"operations":["CREATE","UPDATE"],"resources":["*/*"]}],"sideEffects":"None","timeoutSeconds":30}]}
creationTimestamp: "2024-04-19T08:13:59Z"
generation: 2
labels:
app: webhook
app.kubernetes.io/component: webhook
app.kubernetes.io/instance: cert-manager
app.kubernetes.io/name: webhook
app.kubernetes.io/version: v1.14.4
name: cert-manager-webhook
resourceVersion: "2116"
uid: df35d4ba-3e04-4b61-8bc4-2e4d71396c90
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJ3akNDQVVpZ0F3SUJBZ0lSQVB3REwzd3oxYjA3WCtDaEJ0eFE3SkV3Q2dZSUtvWkl6ajBFQXdNd0lqRWcKTUI0R0ExVUVBeE1YWTJWeWRDMXRZVzVoWjJWeUxYZGxZbWh2YjJzdFkyRXdIaGNOTWpRd05ERTVNRGd4TkRNdwpXaGNOTWpVd05ERTVNRGd4TkRNd1dqQWlNU0F3SGdZRFZRUURFeGRqWlhKMExXMWhibUZuWlhJdGQyVmlhRzl2CmF5MWpZVEIyTUJBR0J5cUdTTTQ5QWdFR0JTdUJCQUFpQTJJQUJHdmxhbzRYUC9sbU0ralRadnJHTnBvMU1uNmcKTUtnVUNGazlqa3JIaHZDbkJsMU1UYms5ckRLOFAwNy9mZmpXY3BJMFlSeVdYb1lJVTlGNGxsV1pQZ0RUWVg3MwoxM3lpb3huSUMrRnVwaXFiOUc1VzBtNzN6S2IvQmZ5RUU1ODRRS05DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trCk1BOEdBMVVkRXdFQi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZIanoybDlNMTNHWllHaEU4amxITG9jRExyYmMKTUFvR0NDcUdTTTQ5QkFNREEyZ0FNR1VDTVFEaC9RRG9FT0lkbXp1TWx1UG1WRUNSenZMSHlUZStoSFBUcHFEbwpiUnVOdGVXN0lvRUh4WExXZVEzQnFJWDZVQmtDTUFtWFlIZHY0VzAzdnpkaGhIQlh0Q0Q2bkE2T3FhSFhnL3R0CkFMT01PZ25vRHR2UFdCb1ZMZkdYNzBSNEZ5Zi8vdz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
service:
name: cert-manager-webhook
namespace: cert-manager
path: /validate
port: 443
failurePolicy: Fail
matchPolicy: Equivalent
name: webhook.cert-manager.io
namespaceSelector:
matchExpressions:
- key: cert-manager.io/disable-validation
operator: NotIn
values:
- "true"
objectSelector: {}
rules:
- apiGroups:
- cert-manager.io
- acme.cert-manager.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- '*/*'
scope: '*'
sideEffects: None
timeoutSeconds: 30
kind: List
metadata:
resourceVersion: ""
# spark-pi.yaml
apiVersion: sparkoperator.k8s.io/v1beta2
kind: SparkApplication
metadata:
name: spark-pi
namespace: default
spec:
type: Scala
mode: cluster
image: apache/spark:3.5.1
imagePullPolicy: Always
mainClass: org.apache.spark.examples.SparkPi
mainApplicationFile: local:///opt/spark/examples/jars/spark-examples_2.12-3.5.1.jar
sparkVersion: "3.5.1"
restartPolicy:
type: Never
volumes:
- name: test-volume
hostPath:
path: /tmp
type: Directory
driver:
volumeMounts:
- name: test-volume
mountPath: /tmp
serviceAccount: spark-operator-spark
labels:
version: "3.5.1"
executor:
instances: 1
volumeMounts:
- name: test-volume
mountPath: /tmp
labels:
version: "3.5.1"kubectl apply -f spark-pi.yaml
$ kubectl get pod spark-pi-driver -o json | jq '.spec.containers[0].volumeMounts'
[
...
{
"mountPath": "/tmp",
"name": "test-volume"
}
]
$ kubectl logs -n spark-operator spark-operator-78bc7dc797-hcvqh | grep webhook.go
I0419 08:19:52.658942 13 webhook.go:152] Starting the Spark admission webhook server
I0419 08:32:48.048584 13 webhook.go:170] Serving admission request
I0419 08:32:48.049801 13 webhook.go:356] Pod spark-pi-1a31418ef57c98d5-exec-1 in namespace default is subject to mutation |
|
Thanks @ChenYi015 for this PR! We will look at merging this as a part of the Kubeflow Spark Operator Release2. |
|
@vara-bonthu I am wondering whether we should use cert-manager as a dependency, may be it would be more suitable that the CA certificate be managed by spark operator. And I am closing this PR and break it into serveral smaller ones, so we can merge it more easily. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
馃洃 Important:
Please open an issue to discuss significant work before you start. We appreciate your contributions and don't want your efforts to go to waste!
For guidelines on how to contribute, please review the CONTRIBUTING.md document.
Purpose of this PR
Close #1977 #1959 #1947
Proposed changes:
Spark operator rbac:
ClusterRolefor spark operator is only used to grant permissions to cluster scoped resources e.g. nodes, leases.Rolefor spark operator in every spark job namespace, and are used to grant permissions to namespace scoped resources e.g. configmaps, services, events.webhook:
hack/gencerts.shwill not be used to generate certificated any more . Instead use cert-manager to manage TLS certificates used in webhook. An self signedIssuerandCertificatewill be created when webhook is enabled.MutatingWebhookConfigurationandValidatingWebhookConfigurationis managed by Helm rather than spark operator. Thus spark operator does not need permissions for webhooks anymore.namespaceSelectorandobjectSelectorfield to webhook configurations, which can filter out spark pods and spark applications in the spark job namespaces.Change Category
Indicate the type of change by marking the applicable boxes:
Rationale
Checklist
Before submitting your PR, please review the following:
Additional Notes