Add support for extensions loading and creation. #399

mscherer · 2021-02-17T18:41:59Z

Using POSTGRESQL_LIBRARIES and POSTGRESQL_EXTENSIONS allow
users to enable extensions (some bundled in the container) without
having to do anything but adding 2 variables.

centos-ci · 2021-02-17T18:42:00Z

Can one of the admins verify this patch?

centos-ci · 2021-02-17T18:42:01Z

Can one of the admins verify this patch?

centos-ci · 2021-02-17T18:42:01Z

Can one of the admins verify this patch?

src/root/usr/share/container-scripts/postgresql/common.sh

phracek · 2021-07-16T09:40:34Z

Why did you remove file 10_pgaudit.conf. shared_preload_libraries = 'pgaudit' is set only for test? It is not needed in the container for users at all?

phracek · 2021-07-16T09:40:41Z

[test]

mscherer · 2021-07-16T09:55:22Z

I removed it because this is used in the test, and would interfer with them.

However, you are right, removing the file make the example incorrect, as the whole README is no longer correct.

A better solution would be to either remove the cp line and keep the example as is, or duplicate the test, so we can test the old way, and the new one.

I will send a updated PR once the tests are finished.

jontrossbach · 2021-07-16T21:14:30Z

I just got done deploying Postgres13 from these changes as a back end to my Discourse instance. They worked for me and allowed me to automate the processes I have been struggling to automate cleanly for a while now. We are going to either need to maintain these changes or have them upstreamed for a Discourse Template we plan to go live somewhere close to next week.

I built it from the instructions in the first README.md. The following might be useful if it was updated to reflect the steps to build on Fedora, e.g. from:

$ git clone --recursive https://github.com/sclorg/postgresql-container.git
$ cd postgresql
$ make build TARGET=rhel7 VERSIONS=13

to something like:

$ git clone --recursive https://github.com/sclorg/postgresql-container.git
$ cd postgresql
# dnf install podman-docker go-md2man distgen
$ make build TARGET=rhel7 VERSIONS=13

mscherer · 2021-07-20T12:47:05Z

Ok so I tried my best to test the old way and the new way.

And for the failure on testing-farm.io, I am unsure what to do, I rebased hoping this would somehow fix the problem.

hhorak · 2021-09-15T09:57:54Z

[test]

mscherer · 2021-09-15T15:06:21Z

Last push should fix the error:

12:17:33 test/run: line 862: DOCKER_EXTRA_ARGS: unbound variable

(which is the only error I see on https://ci.centos.org/job/SCLo-container-postgresql-rh-fedora/292/timestamps/?time=HH:mm:ss&timeZone=GMT+2&appendLog&locale=en_GB )

mscherer · 2022-09-22T14:22:39Z

So I regenerated like the other PR.

mscherer · 2023-02-22T14:32:43Z

I rebased again (conflicts).

phracek · 2023-10-03T08:13:57Z

@mscherer Please rebase it against master including common submodule.

phracek

I am not an postgresql expert. Except postgresql commands everything LGTM.

@fila43 @hhorak Does it make sense to you?

Using POSTGRESQL_LIBRARIES and POSTGRESQL_EXTENSIONS allow users to enable extensions (some bundled in the container) without having to do anything but adding 2 variables.

For easier debugging, it should be pretty easy to switch daemon to log into the /dev/stderr. This adds such a feature by setting POSTGRESQL_LOG_DESTINATION environment variable to /dev/stderr. Requested at sclorg#353

phracek · 2023-10-03T08:58:30Z

Rebased

[test]

Signed-off-by: Petr "Stone" Hracek <phracek@redhat.com>

phracek · 2023-10-03T10:16:43Z

Let's test after generated sources.

[test]

phracek · 2023-10-03T10:39:33Z

Strange behaviour.

There is an error in test/run script:

-----------------------------------------------
Running test run_pgaudit_test (starting at 2023-10-03 10:26:59+00:00) ... 
-----------------------------------------------
c644c8223c59eb64a932a2d1cd2e13e9f3bee4a1c7fd79ec69e1df6e37a788c0
Created container c644c8223c59eb64a932a2d1cd2e13e9f3bee4a1c7fd79ec69e1df6e37a788c0
test/run: line 340: $1: unbound variable
Tests finished with EXIT=1

Tests are passing:

==============================================
Tests were run for image quay.io/sclorg/postgresql-15-c9s:1
==============================================
Test cases results:

 [PASSED] for 'postgresql-container_tests' run_container_creation_tests (00:00:19)
 [PASSED] for 'postgresql-container_tests' run_general_tests (00:00:39)
 [PASSED] for 'postgresql-container_tests' run_change_password_test (00:00:07)
 [PASSED] for 'postgresql-container_tests' run_replication_test (00:00:06)
 [PASSED] for 'postgresql-container_tests' run_doc_test (00:00:00)
 [PASSED] for 'postgresql-container_tests' run_s2i_test (00:00:14)
 [PASSED] for 'postgresql-container_tests' run_test_cfg_hook (00:00:08)
 [PASSED] for 'postgresql-container_tests' run_s2i_bake_data_test (00:00:12)
 [PASSED] for 'postgresql-container_tests' run_s2i_enable_ssl_test (00:00:08)
 [PASSED] for 'postgresql-container_tests' run_upgrade_test (00:00:00)
 [PASSED] for 'postgresql-container_tests' run_migration_test (00:00:23)

Signed-off-by: Petr "Stone" Hracek <phracek@redhat.com>

phracek · 2023-10-03T10:41:19Z

10/root/usr/share/container-scripts/postgresql/common.sh

+  if [ -v POSTGRESQL_EXTENSIONS ]; then
+    for EXT in $POSTGRESQL_EXTENSIONS; do
+      psql -c "CREATE EXTENSION IF NOT EXISTS ${EXT};"
+    done;


Suggested change

done;

done

This is typo, I guess.

phracek

I hit some issues, that needs to be fixed.

phracek · 2023-10-03T10:51:17Z

test/run_test

@@ -897,7 +900,7 @@ run_pgaudit_test()
  # create a dir for config
  config_dir=$(mktemp -d --tmpdir pg-hook-volume.XXXXX)
  add_cleanup_command /bin/rm -rf "$config_dir"
-  cp -r "$test_dir"/examples/pgaudit/* "$config_dir"
+  $DOCKER_EXTRA_ARGS || cp -r "$test_dir"/examples/pgaudit/* "$config_dir"


Are you sure, you want to execute this variable, which is either empty or -e <something>

No, I guess it should have been a "-v DOCKER_EXTRA_ARGS". I guess the test still work despite the code being incorrect and I didn't catch that.

I will correct it

phracek · 2023-10-03T14:47:57Z

@mscherer Please address my comments, if they are relevant.

zmiklank · 2023-11-21T08:33:01Z

Strange behaviour.

There is an error in test/run script:

-----------------------------------------------
Running test run_pgaudit_test (starting at 2023-10-03 10:26:59+00:00) ... 
-----------------------------------------------
c644c8223c59eb64a932a2d1cd2e13e9f3bee4a1c7fd79ec69e1df6e37a788c0
Created container c644c8223c59eb64a932a2d1cd2e13e9f3bee4a1c7fd79ec69e1df6e37a788c0
test/run: line 340: $1: unbound variable
Tests finished with EXIT=1

Tests are passing:

==============================================
Tests were run for image quay.io/sclorg/postgresql-15-c9s:1
==============================================
Test cases results:

 [PASSED] for 'postgresql-container_tests' run_container_creation_tests (00:00:19)
 [PASSED] for 'postgresql-container_tests' run_general_tests (00:00:39)
 [PASSED] for 'postgresql-container_tests' run_change_password_test (00:00:07)
 [PASSED] for 'postgresql-container_tests' run_replication_test (00:00:06)
 [PASSED] for 'postgresql-container_tests' run_doc_test (00:00:00)
 [PASSED] for 'postgresql-container_tests' run_s2i_test (00:00:14)
 [PASSED] for 'postgresql-container_tests' run_test_cfg_hook (00:00:08)
 [PASSED] for 'postgresql-container_tests' run_s2i_bake_data_test (00:00:12)
 [PASSED] for 'postgresql-container_tests' run_s2i_enable_ssl_test (00:00:08)
 [PASSED] for 'postgresql-container_tests' run_upgrade_test (00:00:00)
 [PASSED] for 'postgresql-container_tests' run_migration_test (00:00:23)

@phracek Hi, have you encountered this problem repeatedly? I will create a issue from this so we will not forget it.

This was referenced Feb 17, 2021

WIP: Add 2 helpers for adding extension more easily #398

Closed

Add example how to enable extension #394

Merged

mscherer force-pushed the add_extensions branch from 1382709 to 6ba4cf8 Compare June 22, 2021 16:58

phracek requested changes Jul 16, 2021

View reviewed changes

src/root/usr/share/container-scripts/postgresql/common.sh Show resolved Hide resolved

src/root/usr/share/container-scripts/postgresql/common.sh Show resolved Hide resolved

mscherer force-pushed the add_extensions branch 2 times, most recently from 82b0b9b to 570df13 Compare July 20, 2021 12:45

mscherer force-pushed the add_extensions branch from 570df13 to 031f77f Compare August 31, 2021 12:51

mscherer requested a review from phracek August 31, 2021 12:53

mscherer force-pushed the add_extensions branch from 031f77f to 7b3bb19 Compare September 15, 2021 14:59

mscherer force-pushed the add_extensions branch from 7b3bb19 to 4f25398 Compare September 22, 2022 14:22

mscherer force-pushed the add_extensions branch from 4f25398 to 756d7f9 Compare February 22, 2023 14:32

phracek reviewed Oct 3, 2023

View reviewed changes

mscherer and others added 3 commits October 3, 2023 10:49

Add support for extensions loading and creation.

d7fc4cf

Using POSTGRESQL_LIBRARIES and POSTGRESQL_EXTENSIONS allow users to enable extensions (some bundled in the container) without having to do anything but adding 2 variables.

Regenerate scripts

49349bd

Allow logging to /dev/stderr

f7205f4

For easier debugging, it should be pretty easy to switch daemon to log into the /dev/stderr. This adds such a feature by setting POSTGRESQL_LOG_DESTINATION environment variable to /dev/stderr. Requested at sclorg#353

phracek force-pushed the add_extensions branch from 15bd4a8 to f7205f4 Compare October 3, 2023 08:57

Add generated versions for postgresql-14

939bb4e

Signed-off-by: Petr "Stone" Hracek <phracek@redhat.com>

Remove double testing function

bdf5150

Signed-off-by: Petr "Stone" Hracek <phracek@redhat.com>

phracek reviewed Oct 3, 2023

View reviewed changes

phracek requested changes Oct 3, 2023

View reviewed changes

phracek added enhancement P1 ready for review labels Oct 4, 2023

zmiklank mentioned this pull request Nov 21, 2023

Testsuite seems to be passing, even though a specific tests fails. #537

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for extensions loading and creation. #399

Add support for extensions loading and creation. #399

mscherer commented Feb 17, 2021

centos-ci commented Feb 17, 2021

centos-ci commented Feb 17, 2021

centos-ci commented Feb 17, 2021

phracek commented Jul 16, 2021

phracek commented Jul 16, 2021

mscherer commented Jul 16, 2021

jontrossbach commented Jul 16, 2021 •

edited

mscherer commented Jul 20, 2021

hhorak commented Sep 15, 2021

mscherer commented Sep 15, 2021

mscherer commented Sep 22, 2022

mscherer commented Feb 22, 2023

phracek commented Oct 3, 2023

phracek left a comment

phracek commented Oct 3, 2023

phracek commented Oct 3, 2023

phracek commented Oct 3, 2023

phracek Oct 3, 2023

phracek left a comment

phracek Oct 3, 2023

mscherer Oct 5, 2023

phracek commented Oct 3, 2023

zmiklank commented Nov 21, 2023

Add support for extensions loading and creation. #399

Are you sure you want to change the base?

Add support for extensions loading and creation. #399

Conversation

mscherer commented Feb 17, 2021

centos-ci commented Feb 17, 2021

centos-ci commented Feb 17, 2021

centos-ci commented Feb 17, 2021

phracek commented Jul 16, 2021

phracek commented Jul 16, 2021

mscherer commented Jul 16, 2021

jontrossbach commented Jul 16, 2021 • edited

mscherer commented Jul 20, 2021

hhorak commented Sep 15, 2021

mscherer commented Sep 15, 2021

mscherer commented Sep 22, 2022

mscherer commented Feb 22, 2023

phracek commented Oct 3, 2023

phracek left a comment

Choose a reason for hiding this comment

phracek commented Oct 3, 2023

phracek commented Oct 3, 2023

phracek commented Oct 3, 2023

phracek Oct 3, 2023

Choose a reason for hiding this comment

phracek left a comment

Choose a reason for hiding this comment

phracek Oct 3, 2023

Choose a reason for hiding this comment

mscherer Oct 5, 2023

Choose a reason for hiding this comment

phracek commented Oct 3, 2023

zmiklank commented Nov 21, 2023

jontrossbach commented Jul 16, 2021 •

edited