helm: support encryption config in ceph-csi-cephfs chart #4531
+131
−0
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: {{ .Values.kmsConfigMapName | quote }} | ||
| namespace: {{ .Release.Namespace }} | ||
| labels: | ||
| app: {{ include "ceph-csi-cephfs.name" . }} | ||
| chart: {{ include "ceph-csi-cephfs.chart" . }} | ||
| component: {{ .Values.nodeplugin.name }} | ||
| release: {{ .Release.Name }} | ||
| heritage: {{ .Release.Service }} | ||
| {{- with .Values.commonLabels }}{{ toYaml . | trim | nindent 4 }}{{- end }} | ||
| data: | ||
| config.json: |- | ||
| {{ toJson .Values.encryptionKMSConfig | indent 4 -}} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| {{- if and .Values.rbac.create .Values.rbac.leastPrivileges -}} | ||
| {{- if and .Values.encryptionKMSConfig (eq .Values.encryptionKMSConfig.encryptionKMSType "metadata") .Values.encryptionKMSConfig.secretNamespace .Values.encryptionKMSConfig.secretName -}} | ||
| kind: Role | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| metadata: | ||
| name: {{ include "ceph-csi-cephfs.nodeplugin.fullname" . }} | ||
| namespace: {{ .Values.encryptionKMSConfig.secretNamespace }} | ||
| labels: | ||
| app: {{ include "ceph-csi-cephfs.name" . }} | ||
| chart: {{ include "ceph-csi-cephfs.chart" . }} | ||
| component: {{ .Values.nodeplugin.name }} | ||
| release: {{ .Release.Name }} | ||
| heritage: {{ .Release.Service }} | ||
| {{- with .Values.commonLabels }}{{ toYaml . | trim | nindent 4 }}{{- end }} | ||
| rules: | ||
| # allow to read the encryption key used with the metadata KMS | ||
| - apiGroups: [""] | ||
| resources: ["secrets"] | ||
| verbs: ["get"] | ||
| resourceNames: [{{ .Values.encryptionKMSConfig.secretName | quote }}] | ||
| {{- end -}} | ||
| {{- end -}} | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| {{- if and .Values.rbac.create .Values.rbac.leastPrivileges -}} | ||
| {{- if and .Values.encryptionKMSConfig (eq .Values.encryptionKMSConfig.encryptionKMSType "metadata") .Values.encryptionKMSConfig.secretNamespace -}} | ||
| kind: RoleBinding | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| metadata: | ||
| name: {{ include "ceph-csi-cephfs.nodeplugin.fullname" . }} | ||
| namespace: {{ .Values.encryptionKMSConfig.secretNamespace }} | ||
| labels: | ||
| app: {{ include "ceph-csi-cephfs.name" . }} | ||
| chart: {{ include "ceph-csi-cephfs.chart" . }} | ||
| component: {{ .Values.nodeplugin.name }} | ||
| release: {{ .Release.Name }} | ||
| heritage: {{ .Release.Service }} | ||
| {{- with .Values.commonLabels }}{{ toYaml . | trim | nindent 4 }}{{- end }} | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: {{ include "ceph-csi-cephfs.serviceAccountName.nodeplugin" . }} | ||
| namespace: {{ .Release.Namespace }} | ||
| roleRef: | ||
| kind: Role | ||
| name: {{ include "ceph-csi-cephfs.nodeplugin.fullname" . }} | ||
| apiGroup: rbac.authorization.k8s.io | ||
| {{- end -}} | ||
| {{- end -}} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,6 +2,9 @@ | |
| rbac: | ||
| # Specifies whether RBAC resources should be created | ||
| create: true | ||
| # When possible try and reduce the scope of permission to only give | ||
| # access to resources defined in the config. See the README for more info | ||
| leastPrivileges: true | ||
|
|
||
| serviceAccounts: | ||
| nodeplugin: | ||
|
|
@@ -30,6 +33,20 @@ serviceAccounts: | |
| # netNamespaceFilePath: "{{ .kubeletDir }}/plugins/{{ .driverName }}/net" | ||
| csiConfig: [] | ||
|
|
||
| # Configuration for the encryption KMS | ||
| # yamllint disable-line rule:line-length | ||
| # Ref: https://github.com/ceph/ceph-csi/blob/devel/docs/deploy-cephfs.md#cephfs-volume-encryption | ||
|
There was a problem hiding this comment. this line seems to be too long (ci job) There was a problem hiding this comment. Is it okay for me to break the link into two lines? Or perhaps I should add lint silencer for this line? What is the suggestion? There was a problem hiding this comment. Adding a lint silencer for that line is fine. Breaking the URL over multiple lines isn't that clean. There was a problem hiding this comment. This line directly above the URL should work: |
||
| # Example: | ||
| # encryptionKMSConfig: | ||
| # encryptionKMSType: vault | ||
| # vaultAddress: https://vault.example.com | ||
| # vaultAuthPath: /v1/auth/kubernetes/login | ||
| # vaultRole: csi-kubernetes | ||
| # vaultPassphraseRoot: /v1/secret | ||
| # vaultPassphrasePath: ceph-csi/ | ||
| # vaultCAVerify: "true" | ||
| encryptionKMSConfig: {} | ||
|
|
||
| # Labels to apply to all resources | ||
| commonLabels: {} | ||
|
|
||
|
|
@@ -300,6 +317,18 @@ storageClass: | |
| # If omitted, defaults to "csi-vol-". | ||
| # volumeNamePrefix: "foo-bar-" | ||
| volumeNamePrefix: "" | ||
|
|
||
| # (optional) Instruct the plugin it has to encrypt the volume | ||
| # By default it is disabled. Valid values are "true" or "false". | ||
| # A string is expected here, i.e. "true", not true. | ||
| # encrypted: "true" | ||
| encrypted: "" | ||
|
|
||
| # (optional) Use external key management system for encryption passphrases by | ||
| # specifying a unique ID matching KMS ConfigMap. The ID is only used for | ||
| # correlation to configmap entry. | ||
| encryptionKMSID: "" | ||
|
|
||
| # The secrets have to contain user and/or Ceph admin credentials. | ||
| provisionerSecret: csi-cephfs-secret | ||
| # If the Namespaces are not specified, the secrets are assumed to | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we dont need Role and Role Binding right? we need to have access to secrets in different namespaces as well. i don't see Role https://github.com/ceph/ceph-csi/tree/devel/deploy/cephfs/kubernetes, am i missing anything?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea was to downgrade the
ClusterRoleto just aRolein case the configured secret inencryptionKMSConfigwas local to the namespace. This was done to help with least privileged and limit the access to secret as much as possible. Happy to remove if you think this is overkill. Arguably, users who want least privileged roles can create manage the RBAC themselvesThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ceph-CSI components which reads the Secrets are usually running in a different namespace than the namespace where Secrets for applications have their encryption key. Ceph-CSI needs to read the Secret from a different namespace than where it is running/mounting the PV.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case, it will create the
ClusterRoleautomatically. The logic at line 2 is ifencryptionKMSConfig.secretNamespace == Release.Namespace, then create aRoleinstead of addingsecret:readto theClusterRole.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Happy to scratch that optimisation, and create RBAC myself! But just thought it would be a harmless optimisation to put in
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Yes, a 2nd PR is fine for the RBD enhancement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have slightly reworked the logic behind it to enforce determinism and added a some documentation in the
REAMDE.md. I'd be nice if you could tell me how that looks, and if you like it, I can spin up a PR with the same feature for RBD.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nixpanic it'd be great if you could let me know your though on this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@acolombier @nixpanic is on PTO will be back 1st week of Jun, do you want me to block this PR until he is back to confirm on this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Up to you! If it is ready to be merged, happy to do so too, and I'll wait and see if @nixpanic likes this approach for RBD and I can make the PR then