Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rename and repurpose this repository #2171

Open
ktdreyer opened this issue Nov 13, 2023 · 17 comments
Open

rename and repurpose this repository #2171

ktdreyer opened this issue Nov 13, 2023 · 17 comments

Comments

@ktdreyer
Copy link
Member

ktdreyer commented Nov 13, 2023

The "ceph-container" GitHub project started as an experiment to containerize Ceph and make it integrate with Kubernetes. It has met this goal. I'm opening this ticket to propose how to containerize Ceph in the future.

Years ago, ceph-container provided a lot of extra experimental code (the different "flavors" like "ceph demo", the "sree" app, confd stuff, etc) and supported a wide variety of operating systems (Ubuntu, CentOS, Alpine, etc). Neither of these things are true today. Guillaume has simplified the code now so that it is almost a simple Dockerfile and entrypoint script. We also only publish Ceph images with a single OS (CentOS 8 Stream), greatly simplifying QE effort.

The templating system introduced in 2018 does not provide value today.

I suggest that we have a single Dockerfile at the root of https://github.com/ceph/ceph that will be "the" Ceph container image. (see ceph/ceph#43292). Users who deploy Ceph with cephadm or Rook will use those stable sets of images. This ceph-container GitHub repository will no longer host the Dockerfile for that.

There are a few people in the Ceph community who have discussed maintaining Ubuntu- or Debian-based container images. To provide a common place for these, I suggest renaming this GitHub repo from ceph-container to ceph-containers or alt-containers, and rewrite the README to explain that this is now the location for the non-official or "alternative" container images. You could store simple, untemplated Dockerfiles within subdirectories like ubuntu, debian, etc.

@travisn
Copy link
Member

travisn commented Nov 13, 2023

Great to hear about the simplification. What about the builds that currently come out of the ceph-container repo? For example, in Rook we rely on these tags to test against the latest merges to the various branches:

  • quay.io/ceph/daemon-base:latest-quincy-devel
  • quay.io/ceph/daemon-base:latest-reef-devel

When the images are build from the ceph/ceph repo, can we get these tags also built? Even better if we could just get these tags in the quay.io/ceph/ceph repo, it has always been confusing having the daemon-base repo.

@BlaineEXE
Copy link
Collaborator

@travisn good question.

Personally, I think it makes sense to go one of two ways:

  1. Use quay.io/ceph/ceph for all images including dev builds, OR
  2. Use a new quay.io/ceph/devel library for the dev builds

Probably the latter is best.

I think all the different builds can be handled by the configuration of GitHub actions, combined with any special changes needed in release branches.

In the main branch, the Dockerfile will use the latest main build of Ceph, and the GitHub action will publish to quay.io/ceph/devel:latest-main as well as the commithash version.

In release branches, Dockerfiles will use the latest corresponding release build of Ceph, and the GitHub action will still publish to quay.io/ceph/devel:latest-<release> as well as the commithash version.

When a particular Ceph version on GitHub is tagged as a release, it should be given a tag like v18.2.1 using GitHub's "Releases" feature. When the GitHub action sees a tag like this, it should publish an image with that tag to quay.io/ceph/ceph:<tag>, plus quay.io/ceph/ceph:<tag>-<date> (in addition to the regular commithash and latest-<release> publish to quay.io/ceph/devel). This process can also be used for release candidates with tags like v18.2.1-RC0 if desired.

One of the other things that GitHub actions should probably do is to regularly re-build tagged, released images with the latest base image. The base image (stream8, stream9-minimal, etc.) will have regular, ongoing security fixes that some users may want. A GH action cron that runs daily on v* tags can do this, and there is a github action that should make it easy to only publish new image versions when there are base image changes: https://www.flypenguin.de/2021/07/30/auto-rebuild-docker-images-if-base-image-changes-using-github-actions/

@BlaineEXE
Copy link
Collaborator

I think renaming this repo to alt-containers might be good to signify that its purpose is changed dramatically. I suspect that it is not widely used, and that we won't see many changes or much interest in it after the move Ken is proposing.

@adk3798
Copy link
Collaborator

adk3798 commented Nov 13, 2023

Great to hear about the simplification. What about the builds that currently come out of the ceph-container repo? For example, in Rook we rely on these tags to test against the latest merges to the various branches:

  • quay.io/ceph/daemon-base:latest-quincy-devel
  • quay.io/ceph/daemon-base:latest-reef-devel

When the images are build from the ceph/ceph repo, can we get these tags also built? Even better if we could just get these tags in the quay.io/ceph/ceph repo, it has always been confusing having the daemon-base repo.

I assume however we move forward we'll continue building regular images of the stable branches. We already do daily builds of the pacific onward stable branches to make sure they can be nuilt and used in baseline runs. Those are currently pushed to quay.ceph.io/ceph-ci/ceph:<release> e.g. quay.ceph.io/ceph-ci/ceph:reef and it would be detrimental to lose those. I'm not sure how the naming would be moving forward but I don't see a world where people are happy with us not making regular builds of the stable branches.

@andrewschoen
Copy link
Contributor

I really like this idea. I think it has the potential to simplify processes involved in testing and building ceph containers downstream as well as upstream.

@phlogistonjohn
Copy link

hear, hear! I'm all for simplifying how the container image(s) are built. I've felt for a while that this code is too complex for the result it produces. I might have some quibbles with the exact structure or location for this Containerfile but that can be handled elsewhere.

As for repos and tagging, I feel obligated to mention that ceph produces release containers at quay.io/ceph/ceph as well as ci/testing containers at quay.ceph.io/ceph-ci/ceph. The 'go-ceph' project uses both in it's ci - the former for required tests against major releases and the latter for integration tests against unreleased branches. If the latter is suitable for the rook use case, perhaps no new repos need to be created?

@BlaineEXE
Copy link
Collaborator

I think quay.ceph.io/ceph-ci/ceph is a good option for what I proposed above as quay.io/ceph/devel. I didn't know about those images.

@dmick
Copy link
Member

dmick commented Nov 13, 2023

As you note, this has been proposed before. All the issues above are valid; I'm paranoid about finding all the users of ceph-container, but I think the time is right to simplify (having just spent another few hours trying to understand make macros in ceph-container)

@ira-gordin-sap
Copy link

Great idea! I propose to consider to use https://github.com/GoogleContainerTools/distroless images as base images.

@BlaineEXE
Copy link
Collaborator

FYI: Starting to set up the skeleton of the new Containerfile and github actions builds here: ceph/ceph#54575

@cbodley cbodley mentioned this issue Dec 8, 2023
2 tasks
Copy link

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

Copy link

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.

@kfox1111
Copy link

I wish github didn't close as "completed". :/

@dmick dmick reopened this Apr 18, 2024
@github-actions github-actions bot removed the wontfix label Apr 19, 2024
Copy link

github-actions bot commented May 5, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

Copy link

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.

@ira-gordin-sap
Copy link

Looks like this is work: ceph/ceph#54575, so why to close?

@dmick dmick reopened this May 13, 2024
@github-actions github-actions bot removed the wontfix label May 13, 2024
Copy link

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

@dmick dmick removed the wontfix label May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants