Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dump/restore doesn't work (for versions 1.2.3, 1.2.4 only dump works)! #331

Open
borodaev opened this issue Sep 29, 2022 · 6 comments
Open

Comments

@borodaev
Copy link

borodaev commented Sep 29, 2022

Hello!
Looks like #197 is not finished. For versions 1,2,3 and 1.2.4 (I use pg14 on win 10) I can dump DB with patches but I can't restore
How do you guys live with it?
During restore pcpatch type has a 'lazy', delayed dependency to public.pointcloud_formats table records (to pcid field of the records). This indirect dependency causes error for copy commands execution during pg_restore because pointcloud_formats can be empty at the time when copy is executing.
Sometime restore can work fine if jobs are used and pointcloud_formats gets filled before patches which use corresponding format in this table. For example with attached dump file restore error can be reproduced if no jobs used but for jobs==3 it works fine. So the problem with restore is not related to how DB was created (it can be created with new pgpc version installed with old pgpc version and upgraded then using ALTER EXTENSION)

So I found only one workaround to restore db with patches:
this works only if there are no custom types/functions/tables in public schema:

  1. create new db test

  2. add exts

CREATE EXTENSION postgis;
CREATE EXTENSION pointcloud;
CREATE EXTENSION pointcloud_postgis;
  1. dump data from public only schema
    pg_dump.exe --file "C:\temp\d_public_d.dmp" --host "localhost" --port "5432" --username "postgres" --verbose --format=c --blobs --data-only --schema=public "demolog"
  2. dump all from non-public only schema
    pg_dump.exe --file "C:\temp\d_n_public.dmp" --host "localhost" --port "5432" --username "postgres" --verbose --format=c --blobs --exclude-schema=public "demolog"
  3. restore public data
    pg_restore.exe --host "localhost" --port "5432" --username "postgres" --dbname "test" --data-only --verbose "C:\temp\d_public_d.dmp"
  4. restore the rest
    pg_restore.exe --host "localhost" --port "5432" --username "postgres" --dbname "test" --verbose "C:\temp\d_n_public.dmp"
    db_broken.zip

update
it is possible to restore even with custom data in public - you just need to create full dump and then restore public only schema and then - the rest:
restore.cmd:

@echo off
SET DIR=%cd%
if "%5"=="" (
    echo Arguments: host user password db_name backup_file
	set errorlevel=1
	goto err
)
SET PGPASSWORD=%3
@echo on
cd /d C:\Program Files\PostgreSQL\14\bin
call psql -U %2 -d postgres -c "DROP DATABASE IF EXISTS %4;"
@echo off
if errorlevel 1 (
   goto err
)
@echo on
psql -U %2 -d postgres -c "CREATE DATABASE %4;"
@echo off
if errorlevel 1 (
   goto err
)
@echo on
psql -U %2 -d %4 -c "CREATE EXTENSION postgis; CREATE EXTENSION pointcloud; CREATE EXTENSION pointcloud_postgis;"
@echo off
if errorlevel 1 (
   goto err
)
@echo on
pg_restore.exe --host %1 --port "5432" --username %2 --dbname %4 --schema=public --verbose %5
@echo off
if errorlevel 1 (
   goto err
)
@echo on
pg_restore.exe --host %1 --port "5432" --username %2 --dbname %4 --exclude-schema=public --verbose %5
@echo off
if errorlevel 1 (
   goto err
)
@echo on
echo Success
cd /d %DIR%
Exit /B 0
:err
cd /d %DIR%
Exit /B %errorlevel%

usage example:
restore localhost postgres sa test_db "c:\temp\test.dmp"

env:
SELECT version(), pc_version(), PostGIS_Version()
"PostgreSQL 14.2, compiled by Visual C++ build 1914, 64-bit" "1.2.4" "3.2 USE_GEOS=1 USE_PROJ=1 USE_STATS=1"

@borodaev
Copy link
Author

borodaev commented Oct 1, 2022

Updated description with new (more convenient) workaround

@borodaev
Copy link
Author

borodaev commented Oct 2, 2022

I will try to reproduce it on ubuntu

@borodaev
Copy link
Author

borodaev commented Dec 7, 2022

I've reproduced it on ubuntu as well (tried pg_restore from windows and from ubuntu, also tried from dbeaver)
SELECT version(), pc_version(), PostGIS_Version()
"PostgreSQL 14.6 (Ubuntu 14.6-1.pgdg20.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, 64-bit"
"1.2.4"
"3.3 USE_GEOS=1 USE_PROJ=1 USE_STATS=1"
So now I'm sure that is not possible to simply restore db with patches on windows and unix
How it is possible? How do you live with it?

@autra
Copy link

autra commented Dec 8, 2022

@borodaev it works with pg13 for me. Maybe a regression with pg14? I'll try to test if I have some time.

@borodaev
Copy link
Author

@autra, did you set number of jobs > 1 in restore settings? I will also try on pg13

@autra
Copy link

autra commented Dec 12, 2022

did you set number of jobs > 1 in restore settings?

@borodaev I can't remember, sorry :-/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants