Which container registry use cases are relevant for CSPs?

[matofeder]

Community,

With this thread, I would like to discuss and get your opinion about the open question:

Which container registry use cases are relevant for CSPs?

Some use cases, that were discussed in the registry meeting:

1. As a CSP, I would like to offer a recipe (maintained by SCS) for my customers to deploy the private registry themselves utilizing the CSP infrastructure

2. As a CSP, I would like to offer the whole container registry instance as-a-service per tenant. An administrative user for the registry is then provided for the tenant so that the requester can use said user to gain admin access to the container registry instance.

Possible solution (concept):

• The user should be able to spawn a container registry instance via some simple UI/client with basic inputs (simplified API) e.g. container storage size
  UI/client proposal:
• CSP installs the "container registry" OpenStack plugin (Note: this plugin is not available and should be developed)
• The "container registry" OpenStack plugin offers simplified UI for container registry management as well as OpenStack client extension. It means users could manage their registries via UI or via OpenStack client.
• The "container registry" OpenStack plugin implements SCS container registry adapter, that calls SCS container registry deployment scripts against k8s API
• The UI concept:

• The user gets a container registry URL and admin credentials to access

Backend proposal:

• CSP operates OpenStack with a dedicated project where the SCS k8s cluster lives
• CSP creates a "wire" between UI/client and the dedicated k8s cluster where the user's container registry instances will be spawned

3. As a CSP, I would like to offer a dedicated container registry project as-a-service per user, i.e. a single multi-tenant capable container registry instance is shared across multiple tenants (users). The user is then an administrator of the container registry project (the same approach as in OpenStack and OpenStack projects)

Possible solution (concept):

• The user should be able to create a container registry tenant via some simple UI/client with basic inputs (simplified API) e.g. tenant (project) quota
  UI/client proposal:
• CSP installs the "container registry" OpenStack plugin (Note: this plugin is not available and should be developed)
• The "container registry" OpenStack plugin offers a simplified UI for container registry tenant management as well as OpenStack client extension. It means users could manage their tenants via UI or via OpenStack client.
• The "container registry" OpenStack plugin implements SCS container registry adapter, that calls SCS container registry tenant scripts against container registry API
• The UI concept is the same as in 2
• The user gets a container registry URL and admin credentials to access the project

Backend proposal:

• CSP operates OpenStack with a dedicated project where the SCS k8s cluster lives
• CSP operates a multi-tenant container registry instance in that k8s cluster
• CSP creates a "wire" between UI/client and the container registry instance where the user's container registry tenants will be created

---

Is there any preferable use case that meets your customer's demand? Are there any others?
CSP's opinion on that is highly appreciated (but of course, this discussion is not limited only to CSPs, and any insights/comments are welcomed)

[matofeder]

@joshmue comment taken from the mailing list discussion:

Hi Matej!

Overall, agreeing with these use cases.

A few technical nitpicks (that may be better discussed via Github) below:

The user gets a container registry URL and admin credentials to access
The user gets a container registry URL and admin credentials to access the project

Even though giving out (project) admin credentials would be the first pragmatic step, the goal should be to enable the user to login using their identity from some IdP - via e.g. OIDC.

CSP installs the "container registry" OpenStack plugin (...)
The "container registry" OpenStack plugin offers simplified UI for container registry management as well as OpenStack client extension. It means users could manage their registries via UI or via OpenStack client.

The ability to provision registries just like any other OpenStack resource using the same Web UI, CLI, and API would be indeed the UX that users most likely expect from a managed registry service.
Of course, appending to the OpenStack API would mean that the work done would not benefit CSP's that are not using OpenStack, but other IaaS implementations. That would be blurring the goals written down in https://scs.community/about/ :

Exposing the OpenStack layer is optional from an SCS standardization point of view.
If it is exposed, we however have standards to cover it, so we can deliver compatibility at this layer as well.

In essence, ideally, to deliver compatibility also at the "registry layer", it should not depend on the optional OpenStack API, but should depend on a IaaS-independent API.
Given that there is the perspective for SCS to cover many more different aaS solutions as well, a "SCS API" seems fitting. In the first step maybe a "SCS registry API".
Is that kind of what you mean with "implements SCS container registry adapter"?

Best regards,
Joshua

[matofeder]

@matofeder answer to the previous comment taken from the mailing list discussion:

Hi Joshua,

Thank you for your comments! See my responses below.

The user gets a container registry URL and admin credentials to access
The user gets a container registry URL and admin credentials to access the project
Even though giving out (project) admin credentials would be the first pragmatic step,
the goal should be to enable the user to login using their identity from some IdP - via
e.g. OIDC.

Sure, agree, as you mentioned (project) admin credentials were just proposed as a first step.

CSP installs the "container registry" OpenStack plugin (...)
The "container registry" OpenStack plugin offers simplified UI for container registry management as well as > OpenStack client extension. It means users could manage their registries via UI or via OpenStack client.

The ability to provision registries just like any other OpenStack resource using the same
Web UI, CLI, and API would be indeed the UX that users most likely expect from a managed
registry service.
Of course, appending to the OpenStack API would mean that the work done would not benefit

CSP's that are not using OpenStack, but other IaaS implementations.
That would be blurring the goals written down in https://scs.community/about/ :

Exposing the OpenStack layer is optional from an SCS standardization point of view.
If it is exposed, we however have standards to cover it, so we can deliver compatibility at this layer as well.
In essence, ideally, to deliver compatibility also at the "registry layer", it should not
depend on the optional OpenStack API, but should depend on a IaaS-independent API.

I got your point. It's valid. The proposed solution is just a draft for better visualization of how the whole picture (including UI) could look like.
The proposed solution does not create a dependency on the OpenStack API. The OpenStack layer here is just used as a layer that provides
UI and the Client which will then call the independent "registry layer". The layered architecture could look like this:

UI/Client layer (e.g. OpenStack container registry plugin)
↓
registry layer (e.g. helm install ... --set ...)
↓
infrastructure layer (e.g. k8s-cluster)

The proposed solution wants to be just a reference implementation. IMO the same approach is used in scs/k8s-cluster-api-provider that manages Kubernetes clusters on SCS cloud via cluster-api(CAPI), and currently
only CAPO provider (CAPI OpenStack provider) is implemented.

Given that there is the perspective for SCS to cover many more different aaS solutions as well,
a "SCS API" seems fitting. In the first step maybe a "SCS registry API".
Is that kind of what you mean with "implements SCS container registry adapter"?

Exactly, SCS container registry adapter is a piece of code that a potential OpenStack container registry plugin should have to be able to call "SCS registry API".

Best Regards,

Matej

[saschascherrer]

I agree with these three Use Case descriptions and I'd second a SCS registry API or a MVP SCS API with registry as one use case.

However I'd like to suggest emphasizing that use case 1 and 2 should share as many implementation parts as possible to reduce maintenance overload on the CSP's side. In explanation, the recipe should be the same (or as close to the same as possible) irrespective of whether it is used by the CSP to provide Use Case 2 or 3 or by the end user.
Additionally, I suspect that Use Case 1 would be of least priority as the added benefit for customers over installing your own registry on a (virtual) server or k8s Cluster is the lowest among the listed use cases.

Secondly, I suggest to phrase Use Case 2 a little bit different, as I stumbled over the term user in it's title:

As a CSP, I would like to offer the whole container registry instance as-a-service per tenant. An administrative user for the registry is then provided for the tenant, so that the requester can use said user to gain admin access to the container registry instance.

I would also be happy to see somewhat of a combination of use case 2 and 3 where you can get a dedicated registry for your tenant and have multiple registry projects for different teams. However, this may have the possible disadvantage of having two possibilities of managing registry projects (either as registry admin user or via the "wire" between the UI/client and the container registry) which may interfere with each other.

What are your thoughts on this?

[joshmue]

I would also be happy to see somewhat of a combination of use case 2 and 3 where you can get a dedicated registry for your tenant and have multiple registry projects for different teams. However, this may have the possible disadvantage of having two possibilities of managing registry projects (either as registry admin user or via the "wire" between the UI/client and the container registry) which may interfere with each other.

Rephrasing my understanding of this combination: In an installation/instance that is shared by multiple tenants (so not really "dedicated registry"), one tenant should be able to control multiple "projects".

My naive thought on this: This should be no problem, as a tenant (administrative user) should be capable to provision as many registry projects as they wish through the "SCS registry API".

That is, if the implementation does not create an implicit 1:1 link between tenant and registry project. Also, a tenant (administrator) might get multiple credentials (to be superseded by OIDC login) to manage multiple projects.
Is that the area of problems you are referring to?

[saschascherrer]

Rephrasing my understanding of this combination: In an installation/instance that is shared by multiple tenants (so not really "dedicated registry"), one tenant should be able to control multiple "projects".
That is one implication. However I had something different in mind:

As an organization with multiple teams, you might want a dedicated registry instance for your organization only without any other users on that instance (chances are they have to obey internal policy or regulatory requirements in that regard). However they want to spare the administrative effort to manage the registry as a whole. So they would like to have something like described in use case 3 for each of their teams but without any other organisation on the same registry instance (like in use case 2).

This could be achieved by implementing use case 3 (multi-tenant-capable registry with one project per tenant and additionally adding the possibility to

1. be able to provision multiple registries capable of handling use case 3 and/or
2. add functionality for one organization to have multiple tenants in the registry

[matofeder]

Secondly, I suggest to phrase Use Case 2 a little bit different, as I stumbled over the term user in it's title:

As a CSP, I would like to offer the whole container registry instance as-a-service per tenant. An administrative user for the registry is then provided for the tenant, so that the requester can use said user to gain admin access to the container registry instance.

@saschascherrer, I agree, the title of Use Case 2 has been adjusted as you suggested.

As an organization with multiple teams, you might want a dedicated registry instance for your organization only without any other users on that instance (chances are they have to obey internal policy or regulatory requirements in that regard). However they want to spare the administrative effort to manage the registry as a whole. So they would like to have something like described in use case 3 for each of their teams but without any other organisation on the same registry instance (like in use case 2).

IMO this is related to another open question: Is the shared-storage model in Harbor a limitation for CSPs?. Based on your description, I would say that the shared-storage model in Harbor could be an issue, and there is a use case where CSPs want a dedicated registry instance without any other users (use-case 2), but they do not want to manage the registry as a whole (use-case 3).

I'm wondering whether the approach with dedicated storage for container images per tenant (implemented e.g. in Keppel project) is enough for your proposal.
In other words:
Is it enough for CSPs to have dedicated storage for container images, but still, share some registry core components e.g. users/projects metadata storage (e.g. Postgres DB)? Or this is not an option for CSPs and CSPs want to have a literally dedicated registry instance without any shared parts with other users.