Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

delta backup does not work without $PGDATA in the command line #1585

Open
IgorOhrimenko opened this issue Oct 25, 2023 · 1 comment
Open

Comments

@IgorOhrimenko
Copy link

Database name

postgresql

Issue description

Describe your problem

delta backup does not work without $PGDATA in the command line.
"WALG_DELTA_MAX_STEPS": 7 is set in the .walg.json but when run wal-g backup-push get message:
Features like delta backup are disabled, there might be a performance impact.
If run wal-g backup-push $PGDATA with the same config .walg.json get message Delta backup enabled

Please provide steps to reproduce

WALG_DELTA_MAX_STEPS=1 wal-g backup-push -> Features like delta backup are disabled, there might be a performance impact.
WALG_DELTA_MAX_STEPS=1 wal-g backup-push /var/lib/postgresql/14/cluster-psql-test -> Delta backup enabled

Please add config and wal-g stdout/stderr logs for debug purpose

.walg.json
{
    "WALE_S3_PREFIX": "s3://backup_postgresql_wal-g/cluster-psql-test/14",
    "AWS_ACCESS_KEY_ID": "AWS_ACCESS_KEY_ID",
    "AWS_ENDPOINT": "https://s3",
    "AWS_S3_FORCE_PATH_STYLE": "true",
    "AWS_SECRET_ACCESS_KEY": "pass",
    "AWS_REGION": "reg",
    "PGDATA": "/var/lib/postgresql/14/cluster-psql-test",
    "PGHOST": "/var/run/postgresql",
    "PGPORT": "5432",
    "WALG_DELTA_MAX_STEPS": "7",
    "WALG_PREFETCH_DIR": "/var/lib/postgresql/",
    "WALG_TAR_SIZE_THRESHOLD": "4294967296",
    "WALG_UPLOAD_CONCURRENCY": "1",
    "WALG_COMPRESSION_METHOD": "brotli"
}

This issue has some relatives #1385 and #1248

If you can, provide logs

WALG_DELTA_MAX_STEPS=1 wal-g backup-push
INFO: 2023/10/25 17:02:01.587486 Running remote backup through Postgres connection.
INFO: 2023/10/25 17:02:01.587555 Features like delta backup are disabled, there might be a performance impact.
INFO: 2023/10/25 17:02:01.587558 To run with local backup functionalities, supply [db_directory].
INFO: 2023/10/25 17:02:01.615639 Starting remote backup
INFO: 2023/10/25 17:02:01.721266 Streaming remote backup
INFO: 2023/10/25 17:02:01.732651 Adding data directory
INFO: 2023/10/25 17:02:24.301164 Finishing backup
INFO: 2023/10/25 17:02:24.301199 If wal-g hangs during this step, please Postgres log file for details.
INFO: 2023/10/25 17:02:24.304672 Updating metadata
INFO: 2023/10/25 17:02:24.304711 Uploading metadata
INFO: 2023/10/25 17:02:24.495913 Wrote backup with name base_0000002900000280000000A1
WALG_DELTA_MAX_STEPS=1 wal-g backup-push /var/lib/postgresql/11/cluster-psql-test
INFO: 2023/10/25 17:04:45.001627 LATEST backup is: 'base_0000002900000280000000A1'
INFO: 2023/10/25 17:04:45.055755 Delta backup from base_0000002900000280000000A1 with LSN 280/A1000028.
INFO: 2023/10/25 17:04:45.122151 Calling pg_start_backup()
INFO: 2023/10/25 17:04:45.478021 Delta backup enabled
INFO: 2023/10/25 17:04:45.478051 Starting a new tar bundle
INFO: 2023/10/25 17:04:45.478084 Walking ...
INFO: 2023/10/25 17:04:45.479887 Starting part 1 ...
INFO: 2023/10/25 17:05:00.094650 Packing ...
INFO: 2023/10/25 17:05:00.096655 Finished writing part 1.
INFO: 2023/10/25 17:05:01.702814 Starting part 2 ...
INFO: 2023/10/25 17:05:01.774338 /global/pg_control
INFO: 2023/10/25 17:05:01.812535 Finished writing part 2.
INFO: 2023/10/25 17:05:01.812978 Calling pg_stop_backup()
INFO: 2023/10/25 17:05:05.054343 Starting part 3 ...
INFO: 2023/10/25 17:05:05.054430 backup_label
INFO: 2023/10/25 17:05:05.054439 tablespace_map
INFO: 2023/10/25 17:05:05.057440 Finished writing part 3.
INFO: 2023/10/25 17:05:05.368358 Wrote backup with name base_0000002900000280000000A3_D_0000002900000280000000A1

@x4m
Copy link
Collaborator

x4m commented Mar 30, 2024

Thanks for raising the issue!
I think we should somehow prevent user from doing remote backup when we are running locally and have access to FS...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants