Ceph binaries for ppc64le

[Pensu]

Hi,

Ceph packages for ppc64le are available via distros like Fedora, RHEL etc, but official binaries are not available on download.ceph.com. This issue is to track and provide whatever is required to get ceph ppc64le binaries available.

Please let me know what is required to get this done.

Thanks.

CC: @djgalloway

[ktdreyer]

Hi @Pensu ,

I think Ceph does have some ppc64le hardware, and I'm not sure of the state, and whether that is in the Sepia lab. Maintaining those systems and ensuring they're hooked into Jenkins is going to be an additional sysadmin task.

One option I'd like to consider is re-using the builders that CentOS has hooked into their Koji instance, and increasing the adoption of the CentOS Storage SIG for distributing and consuming Ceph.

[Pensu]

Hi @Pensu ,

I think Ceph does have some ppc64le hardware, and I'm not sure of the state, and whether that is in the Sepia lab. Maintaining those systems and ensuring they're hooked into Jenkins is going to be an additional sysadmin task.

Thanks for the response @ktdreyer. Glad to know ppc64le hardware is already there. Is there any way I can help to get this done?

One option I'd like to consider is re-using the builders that CentOS has hooked into their Koji instance, and increasing the adoption of the CentOS Storage SIG for distributing and consuming Ceph.

Yeah, centos already has some ceph packages available for ppc64le. Any tips on how can we modify these builders to run all the jenkins tests? Plus would this also help us to eventually get ceph-container images for ppc64le in ceph docker registry: https://hub.docker.com/r/ceph/ceph?

[ktdreyer]

I'm not sure we'd modify the CentOS CBS builders themselves to run any tests, but we could probably test Ceph using some of the ppc64le hardware in https://wiki.centos.org/QaWiki/CI/Multiarch

[Pensu]

I'm not sure we'd modify the CentOS CBS builders themselves to run any tests, but we could probably test Ceph using some of the ppc64le hardware in https://wiki.centos.org/QaWiki/CI/Multiarch

@ktdreyer Yeah, sure, this sounds good. So, what should be the next steps for this?

[ktdreyer]

If we already have binaries at http://mirror.centos.org/altarch/7/storage/ppc64le/ceph-nautilus/ , what would be the advantage of building and distributing them at download.ceph.com?

Personally I would like to use CentOS's infrastructure even more so we can scale our build and release efforts. I admit CentOS Storage SIG integration will always be a work-in-progress, but I see the CentOS SIG as a great solution long-term. Adding more architecture platforms is a perfect example of how CentOS's infra can help us.

In terms of ppc64le container images, do those need to be built on ppc64le hardware? @dsavineau is the lead for our container project so I'm cc'ing him here.

[dsavineau]

@ktdreyer yes we need a ppc64le host to build the ppc64le container images. Currently we only have one arm64 host for building ceph centos arm64 container image upstream (jenkins slave).

[Pensu]

@ktdreyer The reason I was asking for download.ceph.com is that the ceph-container repo uses that location to download ceph binaries and build the container images. I think, if the container images can be built using the CentOs repo then we shouldn't need the download site.

@dsavineau Would it be possible to get the ppc64le host from CentOs infra and create ceph-container images for ppc64le?

[dsavineau]

if the container images can be built using the CentOs repo then we shouldn't need the download site.

I don't think that's a good idea.
First because you don't have all ceph releases in CentOS storage SIG. Only jewel, luminous and nautilus and we need to test all realeases. Currently we're building and testing luminous, mimic, nautilus and master/octopus.
Then because honnestly that will be complicated to manage either bits from distro vs upstream in the build system.

Would it be possible to get the ppc64le host from CentOs infra and create ceph-container images for ppc64le?

It seems complicated in the current build system using the Sepia Ceph jenkins infrastruture. And even if we have a dedicated ppc64le host available, we still need to ppc64le packages available upstream (ideally)

[ktdreyer]

When we drop luminous and mimic support in ceph-container, that will satisfy that concern.

Going forward, you're right, we should maintain all of the Ceph release branches in CentOS. I would like to do that in the future.

Maybe we could look at switching for the Octopus release.

[Pensu]

Till the time the infra moves completely to CentOS (in future), is there a way we can make use of ppc64le hosts available in Sepia labs(if they exist!)? Is there anyone who can confirm if the hosts are there and what it would take to get them up and plugged?

@dsavineau When you say we still need to ppc64le packages available upstream , does that mean the packages should be available at download.ceph.com?

@ktdreyer Can we start making use of CentOS infra to create binaries for master/octopus? Let me know however I can help.

[dsavineau]

When you say we still need to ppc64le packages available upstream , does that mean the packages should be available at download.ceph.com?

Exactly

[Pensu]

@ktdreyer So, as @dsavineau said we would need to make binaries available at download.ceph.com. Since CentOS infra is not testing all the releases, should we still go ahead with it? Or should we look at any alternate solution, may be, using/adding ppc64le hosts to Sepia labs? Also, if possible, can we add Jenkins slave, that are not there is Sepia labs and hosted outside?

[ktdreyer]

Personally I do not want to expand the scope of download.ceph.com. I think it would be more worthwhile and sustainable to build this type of thing in CentOS's shared infrastructure.

[Pensu]

@ktdreyer Yeah, makes sense, but not sure if it would be ok for that ceph-container team(Probably @dsavineau can add to that.). Even if we want to move the whole Sepia setup to CentOS infra, any idea how do we get started on that? Or is there any other way we can figure this out?

[Pensu]

@ktdreyer Hi, how can we get started with CentOS infra for ppc64le? Can you please share some starting points to begin with? Or do we need to have some more discussion on this? Thanks.