feat(system-backup): support backup backing image

[ChanYiLin]

ref: longhorn/longhorn#5085

1. Backup all BackingImages after backing up Volumes and their yaml when taking system-backup
   • After backing up Volumes: Because now we already backup BackingImage when backing up Volumes, In this stage, we only backup those BackingImages not being used.
2. Restore BackingImages before restoring Volumes, so the Volumes with BackingImages can restore successfully.
   • Before restoring Volumes: Because Volume with BackingImage creation will fail if there is no BackingImage in the cluster
   • If the BackingImage already exists, update the tag last-skipped-system-restored and last-skipped-system-restored-at
   • If the BackingImage does not exist, restore it by creating restore type BackingImage
     • use Checksum: restore.Status.Checksum, to verify the BackingImage (checksum from the yaml system-backup stored)
     • update the tag last-system-restored and last-system-restored-at

[ChanYiLin]

Manual Test

1. Create a BackingImage and a volume using it
2. Create a system-backup
3. kubectl get lhbbi, lhb -n longhorn-system will see the backup of the Volume and the Backingimage
4. Delete the Volume and the BackingImage
5. Restore the system-backup
6. The BackingImage and the Volume will be restored. The BackingImage source type will be restored and the checksum will be verified.

[c3y1huang]

ref: longhorn/longhorn#5085

1. Backup all BackingImages after backing up Volumes and their yaml when taking system-backup

   • After backing up Volumes: Because now we already backup BackingImage when backing up Volumes, In this stage, we only backup those BackingImages not being used.

2. Restore BackingImages before restoring Volumes, so the Volumes with BackingImages can restore successfully.

   • Before restoring Volumes: Because Volume with BackingImage creation will fail if there is no BackingImage in the cluster

   • If the BackingImage already exists, update the tag last-skipped-system-restored and last-skipped-system-restored-at

   • If the BackingImage does not exist, restore it by creating restore type BackingImage

     • use Checksum: restore.Status.Checksum, to verify the BackingImage (checksum from the yaml system-backup stored)
     • update the tag last-system-restored and last-system-restored-at

Could you also help to update the LEP?

[c3y1huang]

Manual Test

1. Create a BackingImage and a volume using it
2. Create a system-backup
3. kubectl get lhbbi, lhb -n longhorn-system will see the backup of the Volume and the Backingimage
4. Delete the Volume and the BackingImage
5. Restore the system-backup
6. The BackingImage and the Volume will be restored. The BackingImage source type will be restored and the checksum will be verified.

Can this be automated?
https://github.com/longhorn/longhorn-tests/blob/master/manager/integration/tests/test_system_backup_restore.py

[ChanYiLin]

Hi @c3y1huang
Sure, I will update the LEP and the e2e tests.
Thanks for the reminder.

[ChanYiLin]

Hi @c3y1huang
Here is the update of the lep: longhorn/longhorn#8389
PTAL, thanks!

[ChanYiLin]

Hi @c3y1huang
Here is the e2e test: longhorn/longhorn-tests#1864
Thanks!

[mergify]

This pull request is now in conflict. Could you fix it @ChanYiLin? 🙏

[mergify]

This pull request is now in conflict. Could you fix it @ChanYiLin? 🙏

[ChanYiLin]

This pull request is now in conflict. Could you fix it @ChanYiLin? 🙏

Fixed.

[derekbit]

LGTM

[ChanYiLin]

@mergify backport v1.6.x

[mergify]

backport v1.6.x

✅ Backports have been created

• #2824 feat(system-backup): support backup backing image (backport #2710) has been created for branch v1.6.x but encountered conflicts