Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pg_basebackup fails when data-checksums enables on master node. #159

Open
shahidullah79 opened this issue Mar 28, 2024 · 1 comment
Open

Comments

@shahidullah79
Copy link

shahidullah79 commented Mar 28, 2024

pg_basebackup fails when setting up replication.

pg_basebackup -D /var/lib/pgsql/16/standby1 -U replication -p 5434 -Xs -R -P
WARNING: aborting backup due to backend exiting before pg_backup_stop was called
pg_basebackup: error: COPY stream ended before last file was finished
pg_basebackup: removing data directory "/var/lib/pgsql/16/standby1"

Steps to reproduce the issue.

Set PATH variable & Environment variables for PGDATA and archive directories

export PATH=/usr/pgsql-16/bin:${PATH}
export WORKDIR=/var/lib/pgsql/16

MASTER_PGDATA=${WORKDIR}/master
STANDBY1_PGDATA=${WORKDIR}/standby1
STANDBY2_PGDATA=${WORKDIR}/standby2
ARCHIVE_DIR=${WORKDIR}/archive
SQL_FILE_DIR=$PWD
MASTER_PORT=5434
STANDBY1_PORT=5435
STANDBY2_PORT=5436

initdb --data-checksums -D ${MASTER_PGDATA}

Basic configuration of PostgreSQL

cat >> ${MASTER_PGDATA}/postgresql.conf <<EOF
archive_command = 'rsync -a %p ${ARCHIVE_DIR}/%f'
archive_mode = on
wal_level = replica
max_wal_senders = 10
min_wal_size = '80MB'
max_wal_size = '10GB'
hot_standby = on
wal_log_hints = on
listen_addresses = '*'
EOF

cat >> ${MASTER_PGDATA}/pg_hba.conf <<EOF
local replication replication trust
EOF

mkdir -p ${ARCHIVE_DIR}

pg_ctl -D ${MASTER_PGDATA} -l ${WORKDIR}/master.log -o "-p ${MASTER_PORT}" start

Create the replication user and populate encrypted data

psql -p ${MASTER_PORT} -c "CREATE USER replication WITH replication"
psql -p ${MASTER_PORT} -c "ALTER SYSTEM SET shared_preload_libraries = 'pg_tde';"
pg_ctl -D ${MASTER_PGDATA} -l ${WORKDIR}/master.log -o "-p ${MASTER_PORT}" restart

psql -p ${MASTER_PORT} -c "CREATE DATABASE db_tde"
psql -p ${MASTER_PORT} -d db_tde -f ${SQL_FILE_DIR}/tde-sample-schema.sql
psql -p ${MASTER_PORT} -d db_tde -f ${SQL_FILE_DIR}/tde-insert-data.sql

Create the first standby and the start the standby1 server

pg_basebackup -D ${STANDBY1_PGDATA} -U replication -p ${MASTER_PORT} -Xs -R -P

cat >> ${STANDBY1_PGDATA}/postgresql.conf <<EOF
port = ${STANDBY1_PORT}
EOF

pg_ctl -D ${STANDBY1_PGDATA} -l ${WORKDIR}/standby1.log start

Create the second standby and start the server

pg_basebackup -D ${STANDBY2_PGDATA} -U replication -p ${MASTER_PORT} -Xs -R -P

cat >> ${STANDBY2_PGDATA}/postgresql.conf <<EOF
port = ${STANDBY2_PORT}
EOF

pg_ctl -D ${STANDBY2_PGDATA} -l ${WORKDIR}/standby2.log start

@shahidullah79
Copy link
Author

shahidullah79 commented Mar 28, 2024

Shell Script to configure the environment and reproduce the issue.
replication_setup_pg.log

Sql files to populate database with encrypted data.
tde-sample-schema.log
tde-insert-data.log

Master node's server log.
postgresql-Thu.log

@shahidullah79 shahidullah79 changed the title pg_basebackup fails when master node have tde enabled data. pg_basebackup fails when data-checksums enables on master node. Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 📋 Backlog
Development

No branches or pull requests

1 participant