Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copy command does not work as expected #1615

Open
hombre7 opened this issue Jan 10, 2024 · 1 comment
Open

copy command does not work as expected #1615

hombre7 opened this issue Jan 10, 2024 · 1 comment
Labels

Comments

@hombre7
Copy link

hombre7 commented Jan 10, 2024

Database name

postgresql
wal-g-pg v2.0.1

Issue description

copy command does not work as expected

Describe your problem

I need to have another backup copies in different S3. I marked full backup with permanent and try to copy with single command
wal-g copy -b base_000000010000000300000069 -f .walg.json -t dst.json

Command output:

INFO: 2023/12/14 11:35:33.568916 Handle backupname 'base_000000010000000300000069'.
INFO: 2023/12/14 11:35:33.630642 Collecting backup files...
INFO: 2023/12/14 11:35:33.994279 Copied 'basebackups_005/base_000000010000000300000069/metadata.json' from folder '' to 'basebackups_005/base_000000010000000300000069/metadata.json' in fodler ''.
INFO: 2023/12/14 11:35:33.994525 Copied 'basebackups_005/base_000000010000000300000069_backup_stop_sentinel.json' from folder '' to 'basebackups_005/base_000000010000000300000069_backup_stop_sentinel.json' in fodler ''.
INFO: 2023/12/14 11:35:34.038522 Copied 'basebackups_005/base_000000010000000300000069/tar_partitions/pg_control.tar.br' from folder '' to 'basebackups_005/base_000000010000000300000069/tar_partitions/pg_control.tar.br' in fodler ''.
INFO: 2023/12/14 11:35:34.052356 Copied 'basebackups_005/base_000000010000000300000069/tar_partitions/part_007.tar.br' from folder '' to 'basebackups_005/base_000000010000000300000069/tar_partitions/part_007.tar.br' in fodler ''.
INFO: 2023/12/14 11:35:35.651161 Copied '00000001000000030000006D.br' from folder 'wal_005/' to '00000001000000030000006D.br' in fodler ''.
INFO: 2023/12/14 11:35:36.305077 Copied '00000001000000030000006E.br' from folder 'wal_005/' to '00000001000000030000006E.br' in fodler ''.
INFO: 2023/12/14 11:35:36.768259 Copied 'basebackups_005/base_000000010000000300000069/files_metadata.json' from folder '' to 'basebackups_005/base_000000010000000300000069/files_metadata.json' in fodler ''.
INFO: 2023/12/14 11:36:34.611234 Copied 'basebackups_005/base_000000010000000300000069/tar_partitions/part_001.tar.br' from folder '' to 'basebackups_005/base_000000010000000300000069/tar_partitions/part_001.tar.br' in fodler ''.
INFO: 2023/12/14 11:37:40.341172 Copied 'basebackups_005/base_000000010000000300000069/tar_partitions/part_004.tar.br' from folder '' to 'basebackups_005/base_000000010000000300000069/tar_partitions/part_004.tar.br' in fodler ''.
INFO: 2023/12/14 11:37:46.684549 Copied 'basebackups_005/base_000000010000000300000069/tar_partitions/part_003.tar.br' from folder '' to 'basebackups_005/base_000000010000000300000069/tar_partitions/part_003.tar.br' in fodler ''.
INFO: 2023/12/14 11:38:47.861308 Copied 'basebackups_005/base_000000010000000300000069/tar_partitions/part_002.tar.br' from folder '' to 'basebackups_005/base_000000010000000300000069/tar_partitions/part_002.tar.br' in fodler ''.
INFO: 2023/12/14 11:38:47.942463 Copied 'basebackups_005/base_000000010000000300000069/tar_partitions/part_005.tar.br' from folder '' to 'basebackups_005/base_000000010000000300000069/tar_partitions/part_005.tar.br' in fodler ''.
INFO: 2023/12/14 11:38:47.942517 Success copy.

In destination bucket I see basebackups_005 folder but no wal_005 folder, all WAL's archived by wal-g copied in bucket root / directory, so I can't simply restore from new bucket, such as from original.

wal-g backup-list --config dst.json --detail
Output:

name modified wal_segment_backup_start start_time finish_time hostname data_dir pg_version start_lsn finish_lsn is_permanent
base_000000010000000300000069 2023-12-14T11:42:06Z 000000010000000300000069 2023-12-14T10:53:51Z 2023-12-14T10:57:50Z glb-uld-pg02 /var/lib/pgpro/1c-16/data 160001 3/69001048 3/6D969888 true

but
wal-g wal-show --config dst.json --detail
Output: none

Source bucket content:
wal-g st ls

type size last modified name
dir 0 0001-01-01 00:00:00 +0000 UTC basebackups_005/
dir 0 0001-01-01 00:00:00 +0000 UTC wal_005/

Destination bucket content:
wal-g st ls --config dst.json

type size last modified name
dir 0 0001-01-01 00:00:00 +0000 UTC basebackups_005/
obj 2105182 2023-12-14 12:10:20.913 +0000 UTC 00000001000000030000006D.br
obj 3758043 2023-12-14 12:10:21.797 +0000 UTC 00000001000000030000006E.br
obj 2471411 2023-12-14 12:10:22.235 +0000 UTC 00000001000000030000006F.br

Steps to reproduce

Simply try to use wal-g copy and then restore from destination bucket.

@x4m x4m added the bug label Apr 4, 2024
@x4m
Copy link
Collaborator

x4m commented Apr 4, 2024

Thanks for letting us know. This is a bug and we need to fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants