error(s) detected during backup

[j0hannes]

Please provide the following information when submitting an issue (feature requests or general comments can skip this):

1. pgBackRest version: 2.51

2. PostgreSQL version: 16.2

3. Operating system/version - if you have more than one server (for example, a database server, a repository host server, one or more standbys), please specify each: Debian 12

4. Did you install pgBackRest from source or from a package? source

5. Please attach the following as applicable:

   • pgbackrest.conf file(s)

[global]
repo1-path=/u02/pgdata/backups
repo1-retention-full=15
repo1-retention-diff=7
log-path=/u01/app/postgres/local/dmk/log
log-level-file=detail
spool-path=/tmp
start-fast=y
archive-async=y
expire-auto=y
compress-type=bz2
process-max=4


[PG1]
pg1-path=/u02/pgdata/16/PG1
pg1-host=10.65.214.151
pg1-user=postgres
pg1-port=5432
pg2-path=/u02/pgdata/16/PG1
pg2-host=10.65.214.152
pg2-user=postgres
pg2-port=5432
pg3-path=/u02/pgdata/16/PG1
pg3-host=10.65.214.153
pg3-user=postgres
pg3-port=5432

- `postgresql.conf` settings applicable to pgBackRest (`archive_command`, `archive_mode`, `listen_addresses`, `max_wal_senders`, `wal_level`, `port`)

archive_command = '/u01/app/postgres/product/pgbackrest-current/bin/pgbackrest --stanza=PG1 --config=/u01/app/postgres/local/dmk/etc/pgbackrest.conf archive-push /u02/pgdata/16/PG1/pg_wal/%f'
archive_mode = 'on'

- errors in the postgresql log file before or during the time you experienced the issue

No error but with --log-level-console=info we see this:

2024-05-02 16:31:37.724 P00   INFO: last backup label = 20240429-171817F_20240502-141136I, version = 2.51
2024-05-02 16:31:37.724 P00   INFO: execute non-exclusive backup start: backup begins after the requested immediate checkpoint completes
2024-05-02 16:31:38.430 P00   INFO: backup start archive = 0000001F000009BF000000AD, lsn = 9BF/AD000028
2024-05-02 16:31:38.430 P00   INFO: check archive for prior segment 0000001F000009BF000000AC
2024-05-02 16:31:40.915 P00   INFO: execute non-exclusive backup stop and wait for all WAL segments to archive
2024-05-02 16:31:41.117 P00   INFO: backup stop archive = 0000001F000009BF000000AD, lsn = 9BF/AD000138
2024-05-02 16:31:41.126 P00   INFO: check archive for segment(s) 0000001F000009BF000000AD:0000001F000009BF000000AD
2024-05-02 16:31:41.578 P00   INFO: new backup label = 20240429-171817F_20240502-163137I
2024-05-02 16:31:41.738 P00   INFO: incr backup size = 87.8KB, file total = 6937
2024-05-02 16:31:41.738 P00   INFO: backup command end: completed successfully (7246ms)
2024-05-02 16:31:41.738 P00   INFO: expire command begin 2.51: --config=/u01/app/postgres/local/dmk/etc/pgbackrest.conf --exec-id=46312-0804679d --log-level-console=info --log-level-file=detail --log-path=/u01/app/postgres/local/dmk/log --repo1-path=/u02/pgdata/backups --repo1-retention-diff=7 --repo1-retention-full=15 --stanza=PG1
2024-05-02 16:31:41.950 P00   INFO: expire command end: completed successfully (212ms)

- log file in `/var/log/pgbackrest` for the commands run (e.g. `/var/log/pgbackrest/mystanza_backup.log`)

no log file created by the tool

7. Describe the issue:

The backup seems to work as expected, verification doesn't show any issue, but all entries listed with "info" show "error(s) detected during backup".

[dwsteele]

There should be warnings in the log that look like:

invalid page checksums found in file /pg1/base/1/3 at pages 0, 2-4

It looks like you did not redirect stderr so you did not capture them. However, you can look in the logs in /var/log/pgbackrest to find these warnings as well.

What this means is that backup was successful but your database has corruption, i.e. invalid page checksums. You can get a list of corrupted files using the --set option of the info command.

If the corruption is new, you may be able to fix it by restoring an older backup and using PITR to replay from WAL. If the corruption is pre-existing then you'll need to deal with it.