Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make backup copies of configuration files in config_dir #2371

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mbanck
Copy link
Contributor

@mbanck mbanck commented Jul 29, 2022

Before, the .backup files were always stored in the data directory. However, if
there is a distinct configuration directory, it seems logical to put them
there. If config_dir is not configured, it defaults back to the data directory,
so this will not change things for setups where postgresql.conf is in the data
directory.

On the other hand, it will help with setups where postgresql.conf is outside
the data directory and has local changes. In this case, those no longer get
overwritten on boostrap/clone from the primary's backup configuration file that
is streamed to the boostrapped node.

Close #2370

Before, the .backup files were always stored in the data directory. However, if
there is a distinct configuration directory, it seems logical to put them
there. If config_dir is not configured, it defaults back to the data directory,
so this will not change things for setups where postgresql.conf is in the data
directory.

On the other hand, it will help with setups where postgresql.conf is outside
the data directory and has local changes. In this case, those no longer get
overwritten on boostrap/clone from the primary's backup configuration file that
is streamed to the boostrapped node.

Close zalando#2370
@coveralls
Copy link

Pull Request Test Coverage Report for Build 2761120800

  • 2 of 2 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 99.882%

Totals Coverage Status
Change from base Build 2690825560: 0.0%
Covered Lines: 11009
Relevant Lines: 11022

💛 - Coveralls

@CyberDem0n
Copy link
Collaborator

The whole point of creating backups of configuration files is that configs are not actually backed up by some backup tools (for example wal-e). After restoring such a backup we copy these files back.

If config files are stored in the external place things get complex (as usual). It is absolutely normal that the config directory is empty during the bootstrap or replica creation and we want to copy backup files over (current behavior). And your patch will break it.

@mbanck
Copy link
Contributor Author

mbanck commented Sep 6, 2022

The whole point of creating backups of configuration files is that configs are not actually backed up by some backup tools (for example wal-e). After restoring such a backup we copy these files back.

If config files are stored in the external place things get complex (as usual). It is absolutely normal that the config directory is empty during the bootstrap or replica creation and we want to copy backup files over (current behavior).

So what you actually want is to (optionally, or maybe by default, but configurable) copy over the primary's postgresql.base.conf during standby clone if it is not in the data directory, is that correct? And in order to do that, we copy it into the data directory, and let pg_basebackup/the clone method handle it.

And your patch will break it.

Well fair enough, but then Patroni is currently breaking if there's already a local (and different) configuration file. Maybe it should not overwrite existing configuration files? Right now, if there is any change between primary and standby, those get overwritten, resulting in configuration loss.

Would it be ok to make this configurable, i.e. let Patroni know whether we want the .backup files in the data directory or not?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Brittle configuration file backup/restore for external config_dir
4 participants