Remove all but last checkpoints

[tavatir]

Is there a way to remove all but last checkpoints when taking an incremental backup just like the full mode does?

Thank you.

[abbbi]

Leaving the checkpoints behind is part of ensuring consistency of the backup chain, check issue #33

virsh checkpoint-list vm1 --parent
 Name              Creation Time               Parent
----------------------------------------------------------------
 virtnbdbackup.2   2023-01-25 13:59:42 +0100   
 virtnbdbackup.3   2023-01-25 13:59:43 +0100   virtnbdbackup.2

apparently its possible to remove checkpoints using the "virsh checkpoint-delete" function, even if they have a parent,
but i would not recommend this.

[tavatir]

From what I understand, checkpoints are not necessarely related to consistency of the backup files. You need a checkpoint to take an incremental backup but you don't need checkpoints to restore them. Backup files already have the necessary info to restore the chain.

[abbbi]

yes, in the past, libvirt handled checkpoints differently. Checkpoints could either be active or inactive, depending on the state, removing an checkpoint caused an internal merge. Older libvirt versions thus only allowed to remove the root checkpoint. This behavior may have been changed and newer libvirt versions now allow for removing all checkpoints. Im not sure since which libvirt version this is the case. I dont see the leftover checkpoints as a problem currently.

[tavatir]

Another related issue: It denies to take bakeups when there are "foreign" checkpoints to make sure backup chain consistency. However, backup consistency has nothing to do with the checkpoints current live guest have. The live checkpoints are not used when restoring the backups, that would be against the very reason we take backups.

[abbbi]

the checkpoint consistency is not part of the restore, but backup process. what if another application creates an checkpoint that references the last checkpoint created by virtnbdbackup as parent and during the next incremental backup this checkpoint is not backed up? Then i think you have an inconsistency in your backups and during recovery this might result in missing blocks. At least, this is what my understanding is (especially for differential backups)

During restore, the checkpoints have no meaning. Theyre not used or anything.

What "problem" are you trying to solve? Please also consider that there is the "persistent-checkpoint" feature in virtnbdbackup, that tries to keep checkpoint and bitmap information in sync if virtual machines are migrated to different host systems.

The current implementation just wants to make sure the complete chain of backups is comprehensible, also in hinsight of debugging possible issue. Of course it may be possible to tinker with checkpoints not related to virtndbbackup and ignore foreign checkpoints, but im not sure which situations can appear where real inconsistencies can happen. (such as restored filesystems within the qcow images beeing broken because of missing blocks etc ..)

Note: as long as the checkpoints are still existant, it would even be possible to re-construct the backup if it is broken on storage,
without having to start from scratch. It would also be possible to implement an crosscheck function that checks the existing backups against the data still represented within the checkpoint.

If i would change the current code to remove the checkpoints right after the backup, im just not sure which situations can happen where unseen data-loss happens and real shit begins to hit the fan. Also, as long as leaving checkpoints behind does not lead to real performance penalties or things going wrong, i dont see an point to change the current behavior.

[tavatir]

Ok, I now see your point about issue #33.

In my case, I do not need to reconstruct the state of the vm at some point in history, I just need to have the latest state backed up. And I was concerned if having too many checkpoints would slow down the guests (not that I know anything about the internals of checkpoints). I will delete the checkpoints using virsh checkpoint-delete still.

Thanks for the app, it really helps.
(Sorry if my english sounded rude, I didnt mean so)

[abbbi]

no problem, like said, i have not done any testing what happens if you manually delete checkpoints and then attempt an differential backup. The backup will most likely be successful, but will it contain all the data to correctly reconstruct the images? I dont know of any performance issues that appear if many checkpoints exist. After all, its always best to have a good rotation of full backups between incrementals, so it shouldnt be a very big amount of checkpoints kept usually (if you dont intend to do incremental backups for years ..)

After all, an checkpoint (as it beeing an "bitmap" in the qcow image), its just a list of pointers to blocks that have changed, not the changed data itself.