Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API change to improve VolumeGroupSnapshot restore #1068

Merged
merged 7 commits into from May 13, 2024

Conversation

leonardoce
Copy link
Contributor

@leonardoce leonardoce commented Apr 18, 2024

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change

/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:

This PR implements an API change proposed by @xing-yang (see https://docs.google.com/document/d/1NdNwFD5Z64K2heQLYOnojt6Ogulg750BuYIJ6W9cEiM/edit?usp=sharing and the relative conversation in #969).

To ease the review process, this PR has been split into two commits:

  1. a commit changing the API, including the generated files
  2. some minimal changes on the controller and the CSI sidecar to allow the code to compile with the new API.

The information we have in VolumeGroupSnapshot.status.volumeSnapshotRefList has been moved into VolumeGroupSnapshot.status.pvcVolumeSnapshotRefList.volumeSnapshotRef.

The information we have in VolumeGroupSnapshotContent.status.volumeSnapshotContentRefList has been moved into VolumeGroupSnapshotContent.status.volumeSnapshotContentRefList.volumeSnapshotContentName.

The code to set the new fields has already been implemented, and I'm going to open a new PR based on this one.
I hope that it will be helpful for the reviewer.

If you prefer a single PR on multiple commits, just tell me.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

This is my first API-change PR. I followed the process described in client/hack/README.md as carefully as possible, but I encourage you to use special care when reviewing this.

Testing

After this patch, a VolumeGroupSnapshot will be defined as:

apiVersion: groupsnapshot.storage.k8s.io/v1alpha1
kind: VolumeGroupSnapshot
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"groupsnapshot.storage.k8s.io/v1alpha1","kind":"VolumeGroupSnapshot","metadata":{"annotations":{},"name":"new-groupsnapshot-demo","namespace":"default"},"spec":{"source":{"selector":{"matchLabels":{"cnpg.io/cluster":"cluster-example"}}},"volumeGroupSnapshotClassName":"csi-hostpath-groupsnapclass"}}
  creationTimestamp: "2024-04-24T21:09:22Z"
  finalizers:
  - groupsnapshot.storage.kubernetes.io/volumegroupsnapshot-bound-protection
  generation: 1
  name: new-groupsnapshot-demo
  namespace: default
  resourceVersion: "664981"
  uid: e647e38f-d73e-4061-9113-c999d6ebd173
spec:
  source:
    selector:
      matchLabels:
        cnpg.io/cluster: cluster-example
  volumeGroupSnapshotClassName: csi-hostpath-groupsnapclass
status:
  boundVolumeGroupSnapshotContentName: groupsnapcontent-e647e38f-d73e-4061-9113-c999d6ebd173
  creationTime: "2024-04-24T21:09:23Z"
  pvcVolumeSnapshotRefList:
  - persistentVolumeClaimRef: {}
    volumeSnapshotRef:
      name: snapshot-527c30405cb932a46be1207bec23311ded70d6b0ffbfbe534680b8ebd24faea1-2024-04-24-9.9.24
  - persistentVolumeClaimRef: {}
    volumeSnapshotRef:
      name: snapshot-d7734103b40865afaa62b7aa69165c751ae07e9d3a153c62da33bdfe997e1170-2024-04-24-9.9.24
  readyToUse: true

And the corresponding VolumeGroupSnapshotContent would be like:

apiVersion: groupsnapshot.storage.k8s.io/v1alpha1
kind: VolumeGroupSnapshotContent
metadata:
  creationTimestamp: "2024-04-24T21:09:22Z"
  finalizers:
  - groupsnapshot.storage.kubernetes.io/volumegroupsnapshotcontent-bound-protection
  generation: 1
  name: groupsnapcontent-e647e38f-d73e-4061-9113-c999d6ebd173
  resourceVersion: "664972"
  uid: f7eab355-63c6-4f9d-82a7-ef6768e74a17
spec:
  deletionPolicy: Delete
  driver: hostpath.csi.k8s.io
  source:
    volumeHandles:
    - d9c0e0b9-020d-11ef-962a-0abf9c1ed13e
    - d9c87ebe-020d-11ef-962a-0abf9c1ed13e
  volumeGroupSnapshotClassName: csi-hostpath-groupsnapclass
  volumeGroupSnapshotRef:
    apiVersion: groupsnapshot.storage.k8s.io/v1alpha1
    kind: VolumeGroupSnapshot
    name: new-groupsnapshot-demo
    namespace: default
    resourceVersion: "664940"
    uid: e647e38f-d73e-4061-9113-c999d6ebd173
status:
  creationTime: 1713992963010846995
  pvVolumeSnapshotContentList:
  - persistentVolumeRef: {}
    volumeSnapshotContentRef:
      name: snapcontent-527c30405cb932a46be1207bec23311ded70d6b0ffbfbe534680b8ebd24faea1-2024-04-24-9.9.24
  - persistentVolumeRef: {}
    volumeSnapshotContentRef:
      name: snapcontent-d7734103b40865afaa62b7aa69165c751ae07e9d3a153c62da33bdfe997e1170-2024-04-24-9.9.24
  readyToUse: true
  volumeGroupSnapshotHandle: ec95172d-027e-11ef-9db6-961b7658d99d

Does this PR introduce a user-facing change?:

Update API for group snapshots, easing the restore process.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 18, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @leonardoce. Thanks for your PR.

I'm waiting for a kubernetes-csi member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 18, 2024
Change `VolumeSnapshotRefList` in `VolumeGroupSnapshotStatus` to
`PVCVolumeSnapshotRefList`, allowing users to map a `VolumeSnapshot` to
a specific `PersistentVolumeClaim`.

Change `VolumeSnapshotContentRefList` in
`VolumeGroupSnapshotContentStatus` to `PVVolumeSnapshotContentRefList`,
allowing users to map a `VolumeSnapshotContent` to a specific
`PersistentVolume`.

This two changes allow users to map each `VolumeSnapshot` back to the
original PVC it was taken from.
This commit won't set any value in the new fields, but only change the
logic to reproduce the behavior we already had.
@xing-yang
Copy link
Collaborator

/assign

@xing-yang
Copy link
Collaborator

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 18, 2024
// The maximum number of allowed snapshots in the group is 100.
// +optional
VolumeSnapshotRefList []core_v1.ObjectReference `json:"volumeSnapshotRefList,omitempty" protobuf:"bytes,5,opt,name=volumeSnapshotRefList"`
PVCVolumeSnapshotRefList []PVCVolumeSnapshotPair `json:"pvcVolumeSnapshotRefList,omitempty" protobuf:"bytes,6,opt,name=pvcVolumeSnapshotRefList"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should still be 5, not 6.

PVCVolumeSnapshotRefList []PVCVolumeSnapshotPair `json:"pvcVolumeSnapshotRefList,omitempty" protobuf:"bytes,6,opt,name=pvcVolumeSnapshotRefList"`
}

// PVCVolumeSnapshotPair defines a pair of a PVC reference and a Volume Snapshot
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Volume Snapshot reference

PersistentVolumeClaimRef core_v1.ObjectReference `json:"persistentVolumeClaimRef,omitempty" protobuf:"bytes,1,opt,name=persistentVolumeClaimRef"`

// VolumeSnapshotRef is a reference to the VolumeSnapshot this pair is referring to
VolumeSnapshotRef core_v1.ObjectReference `json:"volumeSnapshotRef,omitempty" protobuf:"bytes,1,opt,name=volumeSnapshotRef"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2

// VolumeSnapshotContentRefList is the list of volume snapshot content references
// for this group snapshot.
// PVVolumeSnapshotContentRefList is the list of pairs of PV and
// VolumeSnapshotCOntent for this group snapshot
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PVVolumeSnapshotContentRefList -> PVVolumeSnapshotContentList
VolumeSnapshotCOntent -> VolumeSnapshotContent names

// The maximum number of allowed snapshots in the group is 100.
// +optional
VolumeSnapshotContentRefList []core_v1.ObjectReference `json:"volumeSnapshotContentRefList,omitempty" protobuf:"bytes,5,opt,name=volumeSnapshotContentRefList"`
PVVolumeSnapshotContentRefList []PVVolumeSnapshotContentPair `json:"pvVolumeSnapshotContentRefList,omitempty" protobuf:"bytes,6,opt,name=pvVolumeSnapshotContentRefList"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PVVolumeSnapshotContentRefList -> PVVolumeSnapshotContentList

PersistentVolumeName string `json:"persistentVolumeName,omitempty" protobuf:"bytes,1,opt,name=persistentVolumeName"`

// VolumeSnapshotContentName is the name of the volume snapshot content resource
VolumeSnapshotContentName string `json:"volumeSnapshotContentName,omitempty" protobuf:"bytes,1,opt,name=volumeSnapshotContentName"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2

@leonardoce
Copy link
Contributor Author

Thank you @xing-yang for reviewing this. I should have taken care of your comments.

@xing-yang
Copy link
Collaborator

/assign @bswartz

Copy link
Contributor

@bswartz bswartz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually after reviewing the change, I'm wondering why use object refs at all, if the PVC ref will always be a PVC and the snapshot ref will always be a VolumeSnapshot. Can't we do like the non-namespaced objects do and specify the object names, but omit their kinds? Namespace can be implied by the namespace of the containing object.

@@ -183,79 +183,101 @@ spec:
format: date-time
type: string
type: object
pvcVolumeSnapshotRefList:
description: VolumeSnapshotRefList is the list of PVC and VolumeSnapshot
pair that is part of this group snapshot. The maximum number of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pairs that are part

future.'
type: string
kind:
description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind must be PVC!

future.'
type: string
kind:
description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind must be VolumeSnapshot!

@xing-yang
Copy link
Collaborator

Actually after reviewing the change, I'm wondering why use object refs at all, if the PVC ref will always be a PVC and the snapshot ref will always be a VolumeSnapshot. Can't we do like the non-namespaced objects do and specify the object names, but omit their kinds? Namespace can be implied by the namespace of the containing object.

"Name" might be sufficient for now. I'm wondering we may want to keep the namespace just in case in the future there's namespace transfer support.

Given `Kind` is implied in
`pvcVolumeSnapshotRefList.persistentVolumeClaimRef`
and in `pvcVolumeSnapshotRefList.volumeSnapshotRef`, let's avoid
specifying it.

Signed-off-by: Leonardo Cecchi <leonardo.cecchi@enterprisedb.com>
@leonardoce
Copy link
Contributor Author

I replaced ObjectReferences with LocalObjectReferences and updated the code correspondingly. Thank you @bswartz @xing-yang

@xing-yang
Copy link
Collaborator

LocalObjectReference looks good to me. We should probably use the same construct for PV and VolumeSnapshotContent pairs.
https://github.com/kubernetes/kubernetes/blob/v1.31.0-alpha.0/pkg/apis/core/types.go#L5460
cc @bswartz

Instead of using a field with the resource name, this patch uses
LocalObjectReference in VolumeSnapshot too. This is being done for
simmetry with VolumeGroupSnapshotContent.

Signed-off-by: Leonardo Cecchi <leonardo.cecchi@enterprisedb.com>
@leonardoce
Copy link
Contributor Author

LocalObjectReference looks good to me. We should probably use the same construct for PV and VolumeSnapshotContent pairs.

Nice. I changed the code for this and updated the example in the PR description.

@bswartz
Copy link
Contributor

bswartz commented Apr 24, 2024

LocalObjectReference looks good to me. We should probably use the same construct for PV and VolumeSnapshotContent pairs. https://github.com/kubernetes/kubernetes/blob/v1.31.0-alpha.0/pkg/apis/core/types.go#L5460 cc @bswartz

Non-namespaced objects definitely should not be using namespaces. Assuming the kind is constant, why not just use an object name?

@xing-yang
Copy link
Collaborator

LocalObjectReference looks good to me. We should probably use the same construct for PV and VolumeSnapshotContent pairs. https://github.com/kubernetes/kubernetes/blob/v1.31.0-alpha.0/pkg/apis/core/types.go#L5460 cc @bswartz

Non-namespaced objects definitely should not be using namespaces. Assuming the kind is constant, why not just use an object name?

@msau42 From API design point of view, are there pros and cons of using LocalObjectReference vs just a string here?

Copy link
Contributor

@bswartz bswartz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 2, 2024
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 13, 2024
@xing-yang
Copy link
Collaborator

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 13, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bswartz, leonardoce, xing-yang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 13, 2024
@k8s-ci-robot k8s-ci-robot merged commit 0f88e94 into kubernetes-csi:master May 13, 2024
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants