Feature request: Scheduled backup

[4abhinavjain]

Hi Matthew,

Just wondering if you have considered adding scheduled backup and retention feature. I tried LXDware before but dropped it over https://www.nuber.io/ due to that feature. But I would really like to use LXDware for future project.

Thanks,
Abhinav

[matthewalanpenning]

Hello Abhinav,

Great question. I have considered it and even wrote a solution for it, but then decided against releasing. I found it better in my circumstances to setup the LXD servers with a cron task to automate this. This way if there was any kind of issue with the dashboard connecting to LXD servers (say network outage, or someone shutting down the host running the dashboard) the backups would still continue as scheduled.

I am not opposed to looking into this again. Currently the dashboard has the ability to schedule snapshots, this is because the LXD protocol has this option as a configuration parameter of instances. Unfortunately it does not have this same option for backups.

[4abhinavjain]

Thanks for the update, Matthew.

Scheduled snapshots are a good compromise, but scheduled backups gives extra peace of mind. Nevertheless, if you don't mind posting the cron task based backup approach then it will be very helpful.

[matthewalanpenning]

The backup script will most likely need to be customized as to your organizations needs, but with that said I wrote a quick script in python that would backup all the instances in a list of projects. This could be used as a start to building the solution that you would want. Things to consider would be:

• Which instances will need backed up? What criteria will define which instances are to be backed up on a schedule
• How often do you want to backup the instances? Define your schedule using /etc/crontab.
• What directory should backups to be written to? I would recommend mounting an NFS share or similar to move backups off your LXD server in the event of a failure.
• What naming convention do you want to use for backups? Date and timestamps are helpful.
• How long do you keep backups? A separate scheduled cron task could remove and cleanup outdated backups

With that said here is a python script that can be used as a starting point and having a cron task point to. Save it to a file and call it to run from a schedule in /etc/crontab.

#!/usr/bin/python3

import subprocess
from datetime import date
import os

# Setup vars
backup_dir = '/backups/lxd/'
current_date = str(date.today())
lxd_projects=['default']

# If backup dir does not exist, create it
if not os.path.exists(backup_dir):
   os.makedirs(backup_dir)

# Loop through each project and backup every instance in project
for project in lxd_projects:
  cmd = ['lxc', 'list', '-c', 'n', '-f', 'csv', '--project', project]
  results = subprocess.check_output(cmd, encoding='utf-8').splitlines()
  instances = list(results)

  for instance in instances:
    cmd = ['lxc', 'export', instance, backup_dir + instance + '-' + current_date + '.tar.gz', '--compression', 'gzip', '--project', project]
    results = subprocess.run(cmd)

[4abhinavjain]

Great, Thanks Matthew. Keep up the good work