Installation using a Docker image

[ar3s3ru]

Hey there 👋🏻

Thanks a lot for open-sourcing this!

From the README it seems like we only have 3 ways of creating the database:

1. The install.sh script from the repo,
2. A Ruby gem,
3. An NPM module.

Have you considered creating a Docker image that performs the migration, instead?

I imagine it'd be very similar to the install.sh script, but it would be easier to use in production scenarios, such as Kubernetes job pointed to the PostgreSQL instance used by a service.

Bonus points if the installation is idempotent (running it multiple times with the same version is no-op) and performs updates only once (running the installation on an older version triggers an update, then back to the idempotent behavior). I guess what I'm describing here is a textbook migration 😄

Thoughts?

[sbellware]

@ar3s3ru Maintaining operating system images is deemed beyond the scope of the Message DB project and its social contract with its users.

We maintain the database implementation itself, and we provide some affordances for packaging to benefit the most popular client implementations, which are presently in NodeJS and Ruby. In the future, the responsibility for even these distribution mechanisms are going to be shifted to the open source projects that benefit from them.

The line of what is considered a Message DB project responsibility and what is considered a community contribution responsibility is drawn at the causes and triggers for a need to spend time updating a distribution mechanism:

The Message DB project is intentionally not in the business of keeping OS images up-to-date. They don't change at the same rate and for the same reasons as the core implementation of Message DB itself, and is therefore precluded from the scope of products that the Message DB team assumes responsibility for.

In the end, since it's "just Postgres", and since Postgres is really trivial to get onto a dev machine, and since Message DB is trivial to install into Postgres, there's very little demand for containerized distribution of Message DB.

That said, @juanpaco maintains a publicly-available Docker image of Message DB. I believe he's the official maintainer of the unofficial containerized distribution. It's not something the Message DB team has their hands-on, though.

[ar3s3ru]

Hey @sbellware, I think there's been a misunderstanding. I'm not talking about providing a Docker image with MessageDB inside.

From my original message:

Have you considered creating a Docker image that performs the migration, instead?

I imagine it'd be very similar to the install.sh script, but it would be easier to use in production scenarios, such as Kubernetes job pointed to the PostgreSQL instance used by a service.

I don't know how you folks are setting up the MessageDB migrations in your infrastructure, but I'd imagine you're doing so manually (e.g. using the install.sh script on your staging/live PostgreSQL instances).

Instead of executing a script from a local machine, one could use a Docker image that runs the migration automatically. In a Kubernetes deployment scenario, this could be a simple Job. It would automate the whole thing.

Hope that it's clearer now 😄

[juanpaco]

Hey @ar3s3ru, I like the idea of wrapping the installation as you describe. We're likely soon to be in a place at work where this would be useful to us.

I need to get the docker image I published up to date, so this is probably a good time to do both.

I made an issue at the docker image's repo to track it: SuchSoftware/message-db-docker#2

[sbellware]

@ar3s3ru Yes, that's clearer.

In practice in the wild, different teams and different companies have vastly different approaches to deployment and operations. It's something that varies from project-to-project and based quite a bit on personal preference and matters of opinion.

We don't house any particular deployment code because of the sheer number of options and opinions involved. I've seen the full spectrum from interactive deployment via scripts, to fully automated and containerized deployment based on Helm, Kubernetes, etc. We simply cannot afford to consider maintaining so may options that we ourselves could never personally fully certify for production.

Again, we don't have anything in the core project that we cannot and do not regularly, personally certify. And we cannot take on the responsibility of validating and certifying the vast number of variations in deployment and operations approaches the exist in the wild.

This is why these sorts of things are kept external to Message DB itself, and each specialization maintained by the community member who wishes any particular option to exist in the wild.

[stetsmando]

I was looking for an easy solution to this problem as well. Of course @juanpaco already addressed it. Thanks, Ethan.