Build Docker images in GitHub Actions

[tobiasmcnulty]

Closes #165

What has been done to verify that this works as intended?

1. The images are built via GitHub actions on the caktus fork
2. The images can be pulled from the corresponding GitHub Packages repos

Why is this the best possible solution? Were any other approaches considered?

This PR is an adaptation / scale back from #249. It merely adds images that can be pulled as needed, without changing how the deployment occurs. If there is motivation and a clear path forward, the docker-compose.yml file could be updated (or a corresponding docker-compose.nobuild.yml added) to support using these images from docker-compose.

These images in the meantime will prove useful to non-compose deployment mechanisms, so they don't have to build, push, and host their own images.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

There should be no regression risks since this merely adds the Docker images to GitHub.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

This does not currently require documentation changes, but if there is interest in integrating these images into existing deployment mechanisms, it could be updated for example when the docker-compose.yml file is changed.

Before submitting this PR, please make sure you have:

• verified that any code or assets from external sources are properly credited in comments or that everything is internally sourced

[lognaturel]

Thanks, @tobiasmcnulty! Have you actually tried using the images with your intended configurations and orchestration strategies?

Here's a little bit of context on how things ended up the way that they are. One big goal with Central has been to make it fast and simple enough so that it's possible to run its full stack off a single, inexpensive host. We also try hard to make upgrades, even including important changes like postgres version updates, both transparent to users and approachable to folks who don't have a lot of devops experience (e.g. we've shied away from a management script which can really obfuscate what's going on). Finally, we do want to make it relatively easy for folks to customize. These priorities have left us with a little bit of an unusual setup: we consider the images produced more of an implementation detail, particularly because some configuration does require a rebuild.

I understand people want to do things like scale horizontally and orchestrate containers in ways other than the default. I also understand that the build steps can be time consuming even with the default setup.

All that aside, here are some thoughts on what you have:

• Your approach with the oidc and non-oidc flavors for the nginx image is very interesting. I'm not getting a clear sense of how it would work in the default setup to meet the needs above. Ideally .env is the only file that needs modification for configuration. I think the only way to switch between oidc and non-oidc in a world where the docker compose file uses pre-built images would be to both change the .env file and edit the image name in the docker compose file, right?
• Do you think it really makes sense to build the secrets and enketo images? Those feel like they are implementation details of the way things are orchestrated in the default setup. It seems it might be more appropriate to use the Enketo image directly (https://github.com/enketo/enketo/pkgs/container/enketo/143003614?tag=7.0.0, https://github.com/enketo/enketo/blob/main/Dockerfile) and let whatever's doing the orchestration copy in an appropriate config including secrets.
• Enketo definitely needs a rebuild when changing configured widgets or themes. Those are likely not common customization points but I did want to make that explicit. We've also found that some other configuration changes seem to require rebuilds for reasons I don't quite understand. I don't remember details off the top of my head but will see if I can dig up references. Maybe this doesn't matter for the default setup because we could continue recommending rebuilds where we already do.

[tobiasmcnulty]

Thanks, @tobiasmcnulty! Have you actually tried using the images with your intended configurations and orchestration strategies?

Hi, @lognaturel ! We are running these images on our instance of ODK Central; I just pushed our changes if helpful for context (ignore the pgbouncer stuff at the top). We're still using docker-compose for now, but I believe at least some of these images will be useful in other contexts as well.

Here's a little bit of context on how things ended up the way that they are. One big goal with Central has been to make it fast and simple enough so that it's possible to run its full stack off a single, inexpensive host. We also try hard to make upgrades, even including important changes like postgres version updates, both transparent to users and approachable to folks who don't have a lot of devops experience (e.g. we've shied away from a management script which can really obfuscate what's going on). Finally, we do want to make it relatively easy for folks to customize. These priorities have left us with a little bit of an unusual setup: we consider the images produced more of an implementation detail, particularly because some configuration does require a rebuild.

I understand people want to do things like scale horizontally and orchestrate containers in ways other than the default. I also understand that the build steps can be time consuming even with the default setup.

That's fair, and if you decide that these images stray too far from the course I understand. While my own needs are slightly different, I could see someone making the argument that pulling the pre-built images might lower the barrier to entry even further; as you say, the build steps can be a time consuming part of the default setup.

My concern with updating the docker-compose.yml to use the pre-built images by default is, "what does the migration path look like for existing users?" I might be surprised, for example, if I had customized one of the images, but after an update found my install was using a completely different image pulled from somewhere else. (This is why I ended up pulling those changes out of this PR, and limiting it to just publishing the images for now.)

All that aside, here are some thoughts on what you have:

• Your approach with the oidc and non-oidc flavors for the nginx image is very interesting. I'm not getting a clear sense of how it would work in the default setup to meet the needs above. Ideally .env is the only file that needs modification for configuration. I think the only way to switch between oidc and non-oidc in a world where the docker compose file uses pre-built images would be to both change the .env file and edit the image name in the docker compose file, right?

Correct. I didn't dig into the details here, but from what I could tell in the compose setup, different images are built depending on the build-args and this was the only way to keep the pre-built images consistent with the existing docker-compose setup.

• Do you think it really makes sense to build the secrets and enketo images? Those feel like they are implementation details of the way things are orchestrated in the default setup. It seems it might be more appropriate to use the Enketo image directly (https://github.com/enketo/enketo/pkgs/container/enketo/143003614?tag=7.0.0, https://github.com/enketo/enketo/blob/main/Dockerfile) and let whatever's doing the orchestration copy in an appropriate config including secrets.

My gut says it's better to provide these images too, so long as they are used by docker-compose.yml. There's no requirement that anyone actually use them, and I think you're right that for some use cases it would make more sense to use the Enketo image directly (thank you for the link!).

• Enketo definitely needs a rebuild when changing configured widgets or themes. Those are likely not common customization points but I did want to make that explicit. We've also found that some other configuration changes seem to require rebuilds for reasons I don't quite understand. I don't remember details off the top of my head but will see if I can dig up references. Maybe this doesn't matter for the default setup because we could continue recommending rebuilds where we already do.

It seems totally fair to require rebuilds for custom widgets or themes.

Regarding the "configuration changes seem to require rebuilds" question, I suspect that having a single image that is the source of truth would make this issue go away, for installs that use the pre-built images at least.

[spwoodcock]

+1 it would be very helpful to have pre-built images 👍

@lognaturel would it be useful to have the images multi-architecture, so they work on default AMD64, ARM64 (latest Mac), and others? I assume it's not essential, as Mac devs can build their own images, and ARM64 deployments are rare currently.

It's relatively trivial to add onto the proposed Github workflow, but note it would significantly increase the build time (using QEMU), so best left for release images only. Github also recently released ARM64 runners, but I doubt the cost/benefit would be worth it.

[lognaturel]

+1 it would be very helpful to have pre-built images 👍

What are your thoughts about the secrets and Enketo images as I asked above?

would it be useful to have the images multi-architecture

We generally try not to add things that could be useful without knowing we have a committed user, even when it’s small. Like you say, it seems like something that could be added once someone asks for it.

[spwoodcock]

Agree the pre-built images don't necessarily need ARM64 unless asked.
If we wanted to consider that in the future, then building the backend currently fails on ARM64.

As for bundling the secrets and enketo images:

• If orchestrating with Kubernetes these images are unnecessary. As you say, the config files can just be mounted using a ConfigMap.
• However, the default orchestration is docker compose, and in the spirit of making Central as easy as possible to set up and run, it may be worthwhile building these simple images in GHCR.
• With this said, I don't use Enketo so don't know about the configuration options available. If Enketo accepts environment variables for configuration, then we should definitely use images built directly in that repo instead.