Technical Specification

Here I want to explore how we can generate a static registry for some kind of container (a binary or set of layers) using Github Pages and some storage.

1. API and Organization

For this task, we will do the following:

1. Manifests will be stored in Github Pages.
2. Container binaries will be in Google Storage

Looking at the distribution spec it seems like the primary endpoints for a registry are to serve the tags, (optionally) an exposed catalog, and then blobs. I want to argue that we should remove the blobs from the registry and have them pointed to (with some method, urls?) from the manifests. This means that the registry itself serves to:

• organize the namespace
• provide an API an interface to explore it
• for each entry (container) provide manifests, and tags

And that's really it. This model is akin to a registry "slim" version, because there is no advanced permissions model beyond what Github offers, and updates are completely done via pull requests. In other words, it's a completely simplified and open source registry model. For this to work, we map the following OCI conventions to Github Pages:

Registry

Since the content will be served on Github pages (via the master branch) we can assert that the registry base address is the Github pages address. This means for the repository "singularityhub/container-storage" we use the address "https://singularityhub.github.io/container-storage/.

Namespace

It follows then, that a container namespace are some number of subfolders at the base of the repository. For example, if we want to follow Docker Hub convention with a namespace corresponding to <username>/<reponame> we can create subfolders like "vanessa/greeting"

$ mkdir -p vanessa/greeting

$ tree
├── README.md
└── vanessa           # ( --- vanessa namespace
    └── greeting      # ( --- vanessa/greeting "collection"

This is a nice organization because if I'm browsing the repository, I can find all containers owned by user vanessa in the "vanessa" folder, for example. This means that the complete url for the base of a repository (where we will append other commands to look up types, etc.) corresponds to:

docker://singularityhub.github.io/container-storage/vanessa/greeting

Manifest

The root for image manifests might be found at:

https://singularityhub.github.io/container-storage/vanessa/greeting/manifests

and then have subfolders with the tags of interest, for example. latest:

https://singularityhub.github.io/container-storage/vanessa/greeting/manifests/latest

Let's (for now) create this manually (this would be done programatically)

mkdir -p vanessa/greeting/manifests/latest

In this model we lose "request this version of a manifest" from a repository - the most that might be done is to provide different versions on different branches of the repository.

How would this work? We would want to return json, but we also need the URL to render correctly on Github pages.

Tags

It follows that tags are simply the named folders listed under the manifests folder! It would make sense for each new namespace to have a "permalink" rendered at:

https://singularityhub.github.io/container-storage/vanessa/greeting/tags

That would serve the listing of tags. This isn't hard to do, we simply can add a template to do this too.

mkdir -p vanessa/greeting/tags

with an index.html in that folder that has the listing of tags:

{
  "name": "vanessa/greeting",
  "tags": [
    "latest"
  ]
}

This would also be updated with any changes to the repository.

Blobs

This is the first issue - the blobs are intended to be served by the same base url (of the registry) based on the shasum. My first thought was to create redirect download links, but realizing there is a urls attribute it would be more direct to use this instead. I don't want to store any kind of a blob here, I want to push this responsibility one level down to the storage. The manifest goes directly to the content to download.