Help with empty backups

[coffeverton]

Hi,

i'm trying to set up a backup schedule in my instance.

I can see that the db service is dumping the database every day, but the files on the backup folder are empty.

If I ssh into the docker machine and run backup.sh, then it creates a file with data. But the name is different from the ones that are auto-generated.

Can anyone point me on the right direction to fix this?

Below are some samples from the backup folder. The empty ones are generated by the db service.

-rw-r--r--. 1 root root 0 Dec 21 00:00 backup_bookwyrm_2022-12-21.sql
-rw-r--r--. 1 root root 0 Dec 22 00:00 backup_bookwyrm_2022-12-22.sql
-rw-r--r--. 1 root root 59779747 Dec 5 12:08 backup_fedireads_2022-12-05.sql
-rw-r--r--. 1 root root 87343111 Dec 20 13:12 backup_fedireads_2022-12-20.sql
-rw-r--r--. 1 root root 88309987 Dec 22 13:34 backup_fedireads_2022-12-22.sql

[chdorner]

The file name being different sounds like it's potentially pointing at the issue. Could it be that you are using fedireads as your database name in your setup? i.e. what's the value of POSTGRES_DB in your .env file?

Haven't tested this yet, but I am thinking that when running the backup script through cron in the container it doesn't get that env variable set correctly, vs. when you connect to that docker container and run it manually it does.

[coffeverton]

Hi!
The value of POSTGRES_DB is "fedireads", so it seems that the .env file is not read as you said.

I'll try to find a way to make the backup script read these values from the .env file.

[chdorner]

Just tested this out locally, it looks like the backup script executed by cron doesn't have the env variables setup. For example I added this to the cronfile: * * * * * env > /var/log/env.log, printing out the environment variables every minute. And this is what ended up in file:

HOME=/root
LOGNAME=root
PATH=/usr/bin:/bin
SHELL=/bin/sh
PWD=/root

I've added this as an issue with #2546, I might have some time in the coming days to have a look at this. In the meantime, a workaround could be to use the default database name bookwyrm. I haven't tested it but you might be able to either rename your existing database, or duplicate it with CREATE DATABASE bookwyrm TEMPLATE fedireads.

[coffeverton]

I don't know why my database is named fedireads instead of bookwyrm. But I will wait for your fix. In the meantime, I'll make manual backups.

Thanks!

[chdorner]

I have a proposed fix in #2548.

I think there might be an issue with how changes to the backup.sh script are applied in an environment that has already been running. I had to remove the backups volume when testing this out locally before it picked up changes to that script. Most likely because it is copied to the docker image on build, but later the path is replaced with a volume, so subsequent changes to that file in the docker image aren't reflected on an existing volume.

I wouldn't advise running this on a production instance, but it might work out if you take extra care to backup your backup files first!
I've had to do this for the setup to pick up the new backup.sh script from the docker container:

docker-compose stop

# CAUTION - this deletes the DB container! Data should be preserved since it's on a separate volume, but good to test this out first!
docker-compose rm db

# CAUTION - delete the backup volume. make sure to manually backup the backup files first!
docker volume rm bookwyrm_backups

# start everything back up, building docker images again if there were changes
docker-compose up --build

[chdorner]

@coffeverton the fix has been merged into the production branch, I think if you upgrade your instance as described in the docs your backups should automatically start working.

[mouse-reeve]

It's just in main, not production yet! Should get merged later today or tomorrow though

[chdorner]

The PR #2548 was against production since the Postgres Docker container customizations are only on that branch 🙂

[mouse-reeve]

Oh oh oh! Gotcha thank you, I should have been paying closer attention 😂