feat: implement OpenSchemas types for Container, ContainerImage, etc

[nokome]

We've started implementing a SoftwareEnvironment type (which at this time, doesn't add any properties over schema:SoftwareApplication). We use that in Dockter to generate a Dockerfile (and soon Nix file) from which we build a container.

Meanwhile, I've recently discovered the OpenSchemas initiative which specifically aims to develop schemas for "technology, high performance computing, infrastructure and research computing" and which already has Container, ContainerImage and ContainerRecipe.

OpenSchemas is a sibling of BioSchemas to support schema.org to provide data interoperability for open science. The specifications here are related to technology, high performance computing, infrastructure and research computing. Users are encouraged to use this markup to provide structured understanding to content that drives programmatic discovery.

There is a lot of discussion in various places including shcema.org, OCI and Bioschemas:

• updates for specifications openschemas/specifications#8
• https://github.com/schemaorg/schemaorg/issues/2089
• https://github.com/schemaorg/schemaorg/issues/2059
• Representation of "Container" for Bioschema, and Ontology opencontainers/image-spec#751 (comment)
• Specification for container image, recipe? BioSchemas/specifications#207

[vsoch]

If you want to make it super easy, check out the extractors (Docker containers and Github actions) https://github.com/openschemas/extractors

[nokome]

Closing this for now until there is a specific need for it.

[vsoch]

Thanks @nokome - let me know if I can be of help in the future. It's been very challenging to communicate with the schema.org community and move things forward, so I haven't done any further work here.

[nokome]

Thank you @vsoch. Sorry to hear that you had communication issues with schema.org. I will be reaching out to you (very soon) to get your feedback on our schema.org extensions in this repo.

[vsoch]

For when this is re-opened - I have a few comments!

There are currently three relevant references for containers, namely:

• https://github.com/stencila/schema/blob/master/schema/SoftwareEnvironment.schema.yaml
• https://github.com/stencila/schema/blob/master/schema/SoftwareSession.schema.yaml
• https://github.com/stencila/schema/blob/master/schema/VolumeMount.schema.yaml

And I think a container is fundamentally different, one level up in terms of abstraction from SoftwareEnvironment (e.g., you can imagine a container having a software environment inside. and then when you run the container it could create a VolumeMount (although container is not required for VolumeMount). You mentioned refactoring / changing these shcemas since they aren't widely used, but I would say that it doesn't hurt to have them if needed, and if a use case comes into play that warrants changes, additional work can be done. If you are considering not having these three in the initial release of the library, perhaps there can be a schema drafts or proposals folder alongside the repository to keep them (and then not have them released?)

For containers, the fundamental question, I think, is if it's worth to represent a container image (e.g., referencing a manifest with possibly more than one layer/blob) and/or a container instance, which would be the running thing (with volume mounts, etc.). I think I would implement both, because the use case for the image is to reference something in storage / being published, and the use case for the image would be describing it at runtime. Let me know your thoughts! I'm definitely happy to work on something, even if it just goes into a drafts folder (to await someone that has interest).

[nokome]

After some email discussion with @vsoch we decided to reopen this, rather than create a new issue. Vanessa is a much faster at typing than me so beat me to it :)

We already have three types in this schema, that relate to container images and container runtimes...

SoftwareEnvironment

• Derived from https://schema.org/SoftwareSourceCode (actually it currently erroneously derives from Thing). Which has properties extends, adds and removes to be able to express the derivation and composition of software environments.

• We use it in Nixta as a way to specify Nix environments (see the demo https://asciinema.org/a/KD0z367VL5mBNknueUpqzVGMP)

• We are use it in Dockta as a way of generating a Dockerfile from a SoftwareEnvironment https://github.com/stencila/dockta/blob/master/src/DockerGenerator.ts

• Is similar to OpenSchemas ContainerImage and OCI https://github.com/opencontainers/image-spec/blob/master/spec.md but more abstract in that it is really just meant to define a collection of software packages. So, is it more similar in intent to OpenSchemas ContainerImage?

• Perhaps the best way forward would be to implement OpenSchemas ContainerImage and/or ContainerRecipe here and add a softwareEnvironment property to it.

SoftwareSession

• Similar to OCI Runtime Specification and inspired by things like resource requests and limits in Kubernetes.

• Used in Sparkla e.g. https://github.com/stencila/sparkla/blob/master/host/FirecrackerSession.ts

• Intention is to use it in the Hub for when we start per-user compute sessions.

VolumeMount

• A secondary type needed for defining a software session i.e. the data that the session has access to

---

We have tried to avoid creating new schemas in this project where similar alternatives exist elsewhere. I lean towards using the schemas already defined in OpenSchemas where possible and extending / modifying them where necessary for our particular use case focused on executable documents. We intend to add softwareEnvironment and softwareSession properties to Article, or a new ExecutableArticle type, to define the software environment and compute resources that a document needs.

[vsoch]

So, is it more similar in intent to OpenSchemas ContainerImage?

Hmm, probably not, because a collection of packages != a container image, which can be a binary itself. It would be confusing to have them reference the same thing, because again, a software environment can be found within a container image.

Perhaps the best way forward would be to implement OpenSchemas ContainerImage and/or ContainerRecipe here and add a softwareEnvironment property to it.

Yes! I like this approach. I also think trying to match industry standards (OCI which is one) is the right way to go! What are your thoughts on allowing for some kind of drafts folder (or other method to add drafts?). I could give a shot at adding the ContainerImage and ContainerRecipe.

. I lean towards using the schemas already defined in OpenSchemas where possible and extending / modifying them where necessary for our particular use case focused on executable documents.

I'm of the opinion that if the community is strong here, it's water under the bridge to possibly abandon a previous attempt (openschemas) and retry here. So I don't mind the redundancy, and think it could be better.

[nokome]

All sounds good @vsoch. I've assigned this issue to you and will invite you as a collaborator so you can make a PR.

WRT draft status. Please see the docs folder there is mention there about the custom status property.

Excuse the brevity, on the phone.