What is the recommended approach to enable the plpython3u extension with a customized python3 environment in cloudnative-pg postgresql images?

[cameronraysmith]

I have reviewed this post regarding the creation of custom postgresql container images for CloudNativePG. I have also seen #3672, where it is implied that others are using cloudnative-pg with the plpython3u extension.

I understand one should not plan to update the python environment in the immutable default images. However, it is also clear that these images already include the installation of custom python libraries, barman in particular, as part of the build process.

Would the most highly recommended approach to enabling and working with plpython3u with custom python dependencies be to

1. maintain a fork of https://github.com/cloudnative-pg/postgres-containers with customized Debian/requirements.in from which customized images can be built and sourced as a custom (Cluster)ImageCatalog?

2. More simply, as outlined in the post, one could build an image from a Dockerfile like

FROM ghcr.io/cloudnative-pg/postgresql:16.2-bookworm

USER root

COPY requirements.txt /

RUN set -xe; \
	pip3 install --break-system-packages --no-deps -r requirements.txt;

USER 26

build/push

docker build -t postgresql-plpython3u:16 .
docker push ghcr.io/userorg/postgresql-plpython3u:16 

and, by analogy to the manner in which it is recommended to create PostGIS clusters, use something like the following to create a cloudnative-pg Cluster resource with plpython3u-enabled database(s)

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: plpython3u-cluster-example
spec:
  instances: 3
  imageName: ghcr.io/userorg/postgresql-plpython3u:16
  bootstrap:
    initdb:
      postInitTemplateSQL:
        - CREATE EXTENSION plpython3u;

  storage:
    size: 1Gi

...

Or, is there another recommendation for working with cloudnative-pg clusters whose database(s) require(s) plpython3u postgresql functions?

[obragaa]

Hii Cameron,

You've laid out two solid approaches for integrating the plpython3u extension with custom Python dependencies into CloudNativePG clusters.

1. Maintaining a fork of the repository: This approach involves maintaining a fork of the https://github.com/cloudnative-pg/postgres-containers repository and customizing the Debian/requirements.in file to fit your specific needs. This allows you to build customized images that include all the necessary Python libraries. This is particularly useful if you require a stable and controlled setup for your extensions and dependencies.

2. Direct build via Dockerfile: As you mentioned, building an image directly from a Dockerfile is a more straightforward and perhaps simpler way if the changes are minimal. This involves using the base image ghcr.io/cloudnative-pg/postgresql:16.2-bookworm, installing the required Python dependencies, and then pushing this image to a repository. This approach is quick and efficient for rapid deployments or testing new functionalities.

Regarding creating a cluster resource with plpython3u enabled, the example you provided is appropriate and follows the general recommendation for creating PostGIS clusters, adapted for plpython3u.

If neither of these approaches fully meets your needs, it would be a good idea to discuss this more in-depth with the community or check for additional recommendations directly with the maintainers of CloudNativePG.

I hope this helps clarify your options!

[obragaa]

Particularly, I believe the best approach in your case would be to build directly from a Dockerfile. Here's why I think this might suit your needs well:

1. Simplicity and Speed: Building an image directly from a Dockerfile is a quick and straightforward way to add custom dependencies. This approach allows for rapid iterations and testing, which is ideal for agile development and deployment.

2. Lower Maintenance: Maintaining a fork of a repository can become complex over time, especially when trying to sync with upstream changes. Direct building via a Dockerfile minimizes maintenance efforts and keeps the process more manageable.

3. Flexibility: Using a Dockerfile allows for the flexibility to quickly update or modify dependencies as needed without the complexity of managing a full fork.

4. Control: Even though it's a simpler approach, you still maintain full control over the library versions and other configuration parameters that are crucial for your environment.

5. Adaptation and Compatibility: This method facilitates adaptation as new versions of CloudNativePG or Python libraries are released, allowing for smoother integration and less dependency on major changes in the main repository.

[cameronraysmith]

Yes this sounds reasonable and likely preferable, at least as a first attempt. Thank you for your feedback @obragaa.