Introduced flag to specify when sts deleted, pvc's behavious #4759

CharlesQQ · 2024-03-27T10:25:39Z

What type of PR is this?

/kind bug

What this PR does / why we need it:
Introduced flag to specify when sts deleted, pvc's behavious

Which issue(s) this PR fixes:
Fixes #4750

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


`karmadactl`/`kubectl-karmada`: Introduced `etcd-sts-pvc-delete-policy` flags to specify when sts deleted, pvc's behavious

karmada-bot · 2024-03-27T10:25:49Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign prodanlabs after the PR has been reviewed.
You can assign the PR to them by writing /assign @prodanlabs in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

pkg/karmadactl/cmdinit/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov-commenter · 2024-03-27T11:11:09Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 51.78%. Comparing base (13dafc6) to head (e5f0662).
Report is 23 commits behind head on master.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4759      +/-   ##
==========================================
- Coverage   51.81%   51.78%   -0.03%     
==========================================
  Files         250      250              
  Lines       24983    24991       +8     
==========================================
- Hits        12944    12942       -2     
- Misses      11330    11340      +10     
  Partials      709      709

Flag	Coverage Δ
unittests	`51.78% <100.00%> (-0.03%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

XiShanYongYe-Chang · 2024-03-28T12:52:30Z

pkg/karmadactl/cmdinit/cmdinit.go

 	flags.StringVarP(&opts.EtcdImage, "etcd-image", "", "", "etcd image")
 	flags.StringVarP(&opts.EtcdInitImage, "etcd-init-image", "", kubernetes.DefaultInitImage, "etcd init container image")
 	flags.Int32VarP(&opts.EtcdReplicas, "etcd-replicas", "", 1, "etcd replica set, cluster 3,5...singular")
 	flags.StringVarP(&opts.EtcdHostDataPath, "etcd-data", "", "/var/lib/karmada-etcd", "etcd data path,valid in hostPath mode.")
 	flags.StringVarP(&opts.EtcdNodeSelectorLabels, "etcd-node-selector-labels", "", "", "etcd pod select the labels of the node. valid in hostPath mode ( e.g. --etcd-node-selector-labels karmada.io/etcd=true)")
 	flags.StringVarP(&opts.EtcdPersistentVolumeSize, "etcd-pvc-size", "", "5Gi", "etcd data path,valid in pvc mode.")
+	flags.StringVarP(&opts.EtcdStsPvcDeletePolicy, "etcd-sts-pvc-delete-policy", "", "Retain", fmt.Sprintf("(%s). retain or delete pvc when their stateful set is deleted. Retain, need k8s >= v1.27", strings.Join(kubernetes.SupportedPvcRetentionPolicyType().List(), ",")))


Hi @chaosi-zju @prodanlabs @liangyuanpeng @RainbowMango How do you think this new flag?

XiShanYongYe-Chang · 2024-03-30T03:36:09Z

pkg/karmadactl/cmdinit/kubernetes/deploy.go

@@ -733,6 +735,11 @@ func generateServerURL(serverIP string, nodePort int32) (string, error) {
 }

 // SupportedStorageMode Return install etcd supported storage mode
-func SupportedStorageMode() []string {
-	return []string{etcdStorageModeEmptyDir, etcdStorageModeHostPath, etcdStorageModePVC}
+func SupportedStorageMode() sets.String {


set.String is deprecated, how about use sets.Set[string] to replace it?

how about use sets.Set[string] to replace it?

ok

XiShanYongYe-Chang · 2024-03-30T03:39:38Z

pkg/karmadactl/cmdinit/kubernetes/statefulset.go

+	if i.EtcdStsPvcDeletePolicy == string(appsv1.DeletePersistentVolumeClaimRetentionPolicyType) {
+		stsPvcRetentionPolicy = &appsv1.StatefulSetPersistentVolumeClaimRetentionPolicy{
+			WhenDeleted: appsv1.DeletePersistentVolumeClaimRetentionPolicyType,
+			WhenScaled:  appsv1.RetainPersistentVolumeClaimRetentionPolicyType,


I have a question, I don't understand why this is set up.

why this is set up.

Default policy is appsv1.RetainPersistentVolumeClaimRetentionPolicyType; if set flag, WhenDeleted value shoule be set to appsv1.DeletePersistentVolumeClaimRetentionPolicyType

…behavious Signed-off-by: chang.qiangqiang <chang.qiangqiang@immomo.com>

XiShanYongYe-Chang · 2024-04-02T13:45:18Z

pkg/karmadactl/cmdinit/cmdinit.go

 	flags.StringVarP(&opts.EtcdImage, "etcd-image", "", "", "etcd image")
 	flags.StringVarP(&opts.EtcdInitImage, "etcd-init-image", "", kubernetes.DefaultInitImage, "etcd init container image")
 	flags.Int32VarP(&opts.EtcdReplicas, "etcd-replicas", "", 1, "etcd replica set, cluster 3,5...singular")
 	flags.StringVarP(&opts.EtcdHostDataPath, "etcd-data", "", "/var/lib/karmada-etcd", "etcd data path,valid in hostPath mode.")
 	flags.StringVarP(&opts.EtcdNodeSelectorLabels, "etcd-node-selector-labels", "", "", "etcd pod select the labels of the node. valid in hostPath mode ( e.g. --etcd-node-selector-labels karmada.io/etcd=true)")
 	flags.StringVarP(&opts.EtcdPersistentVolumeSize, "etcd-pvc-size", "", "5Gi", "etcd data path,valid in pvc mode.")
+	flags.StringVarP(&opts.EtcdStsPvcDeletePolicy, "etcd-sts-pvc-delete-policy", "", "Retain", fmt.Sprintf("(%s). retain or delete pvc when their stateful set is deleted. Retain, need k8s >= v1.27", strings.Join(kubernetes.SupportedPvcRetentionPolicyType().UnsortedList(), ",")))


Can we directly provide two flags to set the WhenDeleted and WhenScaled rotation policy type respectively without processing an extra layer and changing the original meaning?

XiShanYongYe-Chang · 2024-04-02T13:46:52Z

What should we do with these flags for Kubernetes versions prior to v1.26?

CharlesQQ · 2024-04-03T09:26:57Z

What should we do with these flags for Kubernetes versions prior to v1.26?

patch ovnerreference to pvc metedata

XiShanYongYe-Chang · 2024-04-03T09:45:03Z

patch ovnerreference to pvc metedata

Sorry, let me rephrase my question. If the version of karmada-apiserver is before 1.27, will the current change have no effect or will it cause the etcd statefulset to report an error?

CharlesQQ · 2024-04-03T09:49:20Z

If the version of karmada-apiserver is before 1.27, will the current change have no effect or will it cause the etcd statefulset to report an error?

sts has no field spec.persistentVolumeClaimRetentionPolicy, so sts can't be created and report an error

XiShanYongYe-Chang · 2024-04-03T10:02:12Z

This behavior may not be expected as we allow the user to set the version of the apiserver:

karmada/pkg/karmadactl/cmdinit/cmdinit.go

Line 121 in 456d099

    
           flags.StringVar(&opts.KubeImageTag, "kube-image-tag", "v1.27.11", "Choose a specific Kubernetes version for the control plane.")

We may need to add some judgment logic.

karmada-bot added kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/bug Categorizes issue or PR as related to a bug. labels Mar 27, 2024

karmada-bot requested review from chaosi-zju and prodanlabs March 27, 2024 10:25

karmada-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 27, 2024

CharlesQQ changed the title ~~fix(/karmadactl): Introduced flag to specify when sts deleted, pvc's…~~ Introduced flag to specify when sts deleted, pvc's behavious Mar 27, 2024

CharlesQQ force-pushed the cascade-deletion-pvc branch from fdd4571 to ebcf41d Compare March 27, 2024 10:57

karmada-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 27, 2024

CharlesQQ force-pushed the cascade-deletion-pvc branch from ebcf41d to 0915630 Compare March 27, 2024 11:21

XiShanYongYe-Chang mentioned this pull request Mar 28, 2024

fix: kubectl karmada deinit doesn't cascade deletion of pvc resources #4750

Open

XiShanYongYe-Chang reviewed Mar 28, 2024

View reviewed changes

XiShanYongYe-Chang reviewed Mar 30, 2024

View reviewed changes

fix(/karmadactl): Introduced flag to specify when sts deleted, pvc's …

e5f0662

…behavious Signed-off-by: chang.qiangqiang <chang.qiangqiang@immomo.com>

CharlesQQ force-pushed the cascade-deletion-pvc branch from 0915630 to e5f0662 Compare April 1, 2024 07:16

XiShanYongYe-Chang reviewed Apr 2, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduced flag to specify when sts deleted, pvc's behavious #4759

Introduced flag to specify when sts deleted, pvc's behavious #4759

CharlesQQ commented Mar 27, 2024 •

edited

karmada-bot commented Mar 27, 2024

codecov-commenter commented Mar 27, 2024 •

edited

XiShanYongYe-Chang Mar 28, 2024

XiShanYongYe-Chang Mar 30, 2024

CharlesQQ Apr 1, 2024

XiShanYongYe-Chang Mar 30, 2024

CharlesQQ Apr 1, 2024 •

edited

XiShanYongYe-Chang Apr 2, 2024

XiShanYongYe-Chang commented Apr 2, 2024

CharlesQQ commented Apr 3, 2024

XiShanYongYe-Chang commented Apr 3, 2024

CharlesQQ commented Apr 3, 2024

XiShanYongYe-Chang commented Apr 3, 2024

Introduced flag to specify when sts deleted, pvc's behavious #4759

Are you sure you want to change the base?

Introduced flag to specify when sts deleted, pvc's behavious #4759

Conversation

CharlesQQ commented Mar 27, 2024 • edited

karmada-bot commented Mar 27, 2024

codecov-commenter commented Mar 27, 2024 • edited

Codecov Report

XiShanYongYe-Chang Mar 28, 2024

Choose a reason for hiding this comment

XiShanYongYe-Chang Mar 30, 2024

Choose a reason for hiding this comment

CharlesQQ Apr 1, 2024

Choose a reason for hiding this comment

XiShanYongYe-Chang Mar 30, 2024

Choose a reason for hiding this comment

CharlesQQ Apr 1, 2024 • edited

Choose a reason for hiding this comment

XiShanYongYe-Chang Apr 2, 2024

Choose a reason for hiding this comment

XiShanYongYe-Chang commented Apr 2, 2024

CharlesQQ commented Apr 3, 2024

XiShanYongYe-Chang commented Apr 3, 2024

CharlesQQ commented Apr 3, 2024

XiShanYongYe-Chang commented Apr 3, 2024

CharlesQQ commented Mar 27, 2024 •

edited

codecov-commenter commented Mar 27, 2024 •

edited

CharlesQQ Apr 1, 2024 •

edited