Global Built-In Registry - Using Harbor

[rcarmo]

I've been poring over past (closed) issues, and see no explicit mention of this, so thought it best to ask even if it's only to have it filed away for future reference:

• Even given that running a registry can be considered an entirely different proposition from orchestrating containers, wouldn't it make sense for Rancher to have a built-in registry, even if there was no UI for it?

This would be an added value (i.e., deploy Rancher, have a complete solution) and might also do away with a fair amount of issues of various kinds regarding trusting private registries.

[ebuildy]

Docker registry runs with Docker very easily, I am not sure of the plus value, maybe with some hooks that for instance, could restart Rancher services when you push a new image.

[rcarmo]

"very easily" doesn't really go too well with registry 2.0. Setting up the required certificates and the backing store is mandatory if you intend to actually use instead of just playing around with it, so I think there's plenty of added value there.

It's not just docker run registry:2, it actually requires some sane defaults, plus each Rancher host has to be manually configured to trust a new registry.

[JamesBarwell]

I've just been setting up a registry and for me there would be a lot of value if Rancher could:

• set up multiple redudant registries across physical hosts with a shared data store and load balancing
• simplify addition of the certificate and key through the UI
• automatically configure itself to consume images from the registry service

[nicolasmery]

+1 @JamesBarwell

[robzhu]

+1

[ibuildthecloud]

Keep the +1's coming :). If we get enough requests will do something along the lines of what @JamesBarwell suggested.

FYI @cloudnautique

[fentas]

+1

[lavvy]

Yeah +1. I see this feature almost like resting on a distributed volume
concept being defaulted for rancher. Cos there still many apps that may
come in the future that will still make use of this concept
On Aug 27, 2015 1:21 PM, "Jan Guth" notifications@github.com wrote:

+1

—
Reply to this email directly or view it on GitHub
#1738 (comment).

[dx9]

+1

[EliSnow]

+1

[willseward]

👍

[Rucknar]

+1

[blackjid]

👍

[cusspvz]

tl;dr

Don't see built-in private registry as an option just because you have to setup storage. That means that you would need to have some configuration on admin panel before starting it.

But since I've read the issue #1272 a few minutes ago, it seems a better solution for this case.

EDIT: just because it seems you could setup options on templates, IMHO that discards the need of built that in.

[vincent99]

Templates still need to ultimately deploy a docker image, which comes from a registry, and people don't necessarily want their image in public DockerHub...

[fentas]

@cusspvz I do not see quite the connection between docker registry and templates. A built in docker registry in rancher would mean to me

• easy self managed (private) docker registry. (no hassle with certs etc.)
• no need of external traffic (image pull) > faster.
• integrated functionality like restarting of running containers if new images are available. (maybe easy configuration with container labels)
• and all what JamesBarwell mentioned.

[cusspvz]

@vincent99 yep, but registry image is public, which mean you could have a template that creates a private registry service on your hosts, allows to configure storage (to prevent issues with later reboots) and its up and running as it was built-in. You could even launch multiple private registries with for different teams or stacks. :)

[cusspvz]

@fentas

easy self managed (private) docker registry. (no hassle with certs etc.)

That should be useful, but rancher allows you to add any registry right?

no need of external traffic (image pull) > faster.

My idea covers that also, well, as it runs as a normal service, you could also say in which hosts should the registry run.

integrated functionality like restarting of running containers if new images are available. (maybe easy configuration with container labels)

That should be triggered by the CD software, not by the registry. Example: tests workflow could vary between companies, if a company needs to push images so it could be tested by worker services, this would be a hassle. So, Continuous Development Software should be the one who triggers an upgrade (not difficult if you use rancher-composer to do a rolling upgrade after testing and pushing the image to the registry).

and all what JamesBarwell mentioned.
set up multiple redudant registries across physical hosts with a shared data store and load balancing

Allowable by the templating feature

simplify addition of the certificate and key through the UI
automatically configure itself to consume images from the registry service

A fork of registry could be created for rancher to do just that

[kaos]

+1 :D

[gordontyler]

+1!

[dhax]

+1

[see0]

+1

[ajagnanan]

+1

[gbisheimer]

There is a nice project to manage private docker registry v2 token authentication named Portus. May be it can be integrated into rancher.

[mishak87]

@gbisheimer I quickly scoped Protus and it has several configuration issues which prevent it running in production mode just from docker-compose.yml and one nasty bug when tagging images.

👍

To keep it simple I would suggest registry:2 behind nginx proxy with htpasswd read/write ACL and use SSL certs stored in Rancher.

[Tails]

+1

[willseward]

I agree with @mishak87. I tried standing Portus up on rancher, and it would require a few modifications in order to run properly w/o initialization by the user. It also locked up one of my hosts by filling up the HD. I'm not sure why though; I didn't do a postmortem.

[gbisheimer]

@mishak87, before finding Portus I also found this blog that explains how to setup docker_auth to use the new docker registry v2 token-authentication. This is a nice feature that allows to have more fine-grained access control to the registry images.

The nice thing about Portus is that it uses token authentication and manages namespaces, teams and users, and allow to limit the access level (owner, collaborator, viewer) of different team members to the registry. Sure it's not usable in production right now, and also it may need to be ported from Rails before including it in Rancher, but it's a starting point.

Registry proxy cache is also a nice feature that should be included

[deniseschannon]

Acceptance Criteria for Alpha

[ ] Have a HA cluster with local cluster enabled, add a PV using NFS to the local cluster. Enable Harbor using the PV option. - Only Admin
[ ] Enable Harbor, selecting the S3 option - Only Admin
[ ] Add images to Harbor - Cluster Owner/Cluster Member and Project Owner/Project Member
[ ] Add username and password to the global registry - Cluster Owner/Cluster Member and Project Owner/Project Member
[ ] Deploy workloads using the private images - Cluster Owner/Cluster Member and Project Owner/Project Member

[sowmyav27]

verified on rancher:master

1. Rancher setup - HA with local cluster.
2. Enabled Harbor using

• Storage Backend for Registry - S3
• Config Database Type - external - Postgres
• Config Redis Type - external - Redis

3. Able to enable Harbor Registry by Admin.
4. Other user, other than Global Admin - Cluster Owner, Cluster member and Project Owner - Able to create an account in Harbor UI, push images to the Harbor registry and deploy workload using these private images

[jiaqiluo]

verified on rancher: master

• Enabled Harbor on the Rancher HA setup using self-signed certs
• Enabled Harbor by using EBS storage class
• as a project member, using Global Registry on EKS, AKS and imported GKE cluster

Known issue:

• using NFS to for PV in the local cluster: Global Registry - cannot work properly with NFS as storage option #20681

[jiaqiluo]

verified on rancher: master

• Enabled Harbor on the Rancher HA setup using Let's Encrypt

[jiaqiluo]

verified on rancher: master

enable Harbor using EBS as the Storage Class

• Add images to Harbor - Cluster Owner/Cluster Member and Project Owner/Project Member
• Add username and password to the global registry - Cluster Owner/Cluster Member and Project Owner/Project Member
• Deploy workloads using the private images - Cluster Owner/Cluster Member and Project Owner/Project Member