Support references to component objects across specification documents

[jsenko]

Currently, reference objects in Apicurio can only point to the same document they are used in. Implement features to allow for referencing component objects from another documents.

Considerations:

1. Location

• Reference component objects from another specification within Apicurio
• Reference documents hosted on another server
• Allow for component objects to be managed separately?

2. Editor

• User should be able to easily fill in references to documents and objects Apicurio knows about (without opening them, auto-complete?)

3. Validation

• Apicurio should validate referenced objects
• What happens if the referenced document is updated or deleted?

Similar/Included issues:

• Linking External Documents #401 Linking External Documents

[EricWittmann]

Use Case

The main use case that this will solve is to give users the ability to centralize their data type definitions, perhaps in a single file (or a set of files). These data types can then be leveraged in multiple API definitions by referencing their central location.

Considerations

In order to support external references, a number of enhancements are needed. Here is a list in no particular order:

• The data model library (currently oai-ts-core but soon to be apicurio-data-models) must be enhanced to support a pluggable reference resolver. Currently these libraries are able to resolve internal references only. Instead, a consumer of the library must be able to provide a custom reference resolver so that external references can be successfully resolved.

• Apicurio Studio should support a "data type only" style resource, with an editor that is focused on designing data types. This allows a more specific workflow for creating/managing data types independently from API designs.

• When setting a type in the API Design editor, the tooling needs a way to select types from external resources. We need to consider how best to display these external types to the user for selection. It may be that we need a way for API Designs to "import" other resources/files so that the UI can narrow down the choices for the user and sensibly display them. Keep in mind that the editor would need to fetch and parse any external resources in order to display their available data types in some sort of selection UI control.

• We may need to decide whether in-progress data type resources can be used as the source for external references in an API design. In other words, should it be possible to reference "snapshot" data types in an API design? Or should we require a Data Type resource to be "final" before it can be used as a source for external references. If the former, then we can't "finalize" an API design until its dependencies are also finalized. Note: this whole concept requires us to implement finalizing resources in Apicurio!

• We will need to decide what the $ref format will be when referencing Apicurio resources. Currently any API Design (or theoretical Data Types resource) has a unique ID, but it's an opaque ID that is generated upon creation. That ID is currently the only unique way to identify a resource for resolution purposes. Once a resource is finalized (if we support such a concept) then we could perhaps grant the resource a more human friendly identifier and then use that in the $ref URI. Whatever we do, it should be noted that when publishing a resource to an API registry (another future feature) any $references in the API Design will need to be resolvable by the API registry in some sensible way.

Here is some information about the allowed values for a $ref:

Even though the value of a $ref is a URI, it is not a network locator, only an identifier. This means that the schema doesn’t need to be accessible at that URI, but it may be. It is basically up to the validator implementation how external schema URIs will be handled, but one should not assume the validator will fetch network resources indicated in $ref values.

More information about references can be found in the OpenAPI specification and the JSON Reference RFC.

[gezhl]

Would it be possible to prioritise implementation of references to DataTypes from another specification within Apicurio (rather than external references initially)?

The reason I say this is that some cross-reference functionality is necessary for any non-trivial usage of Swagger. For example, we are implementing various services that share data types (e.g. addresses, people etc.) and would like to use Apicurio for the specifications for all of these. However, at the moment we cannot because of the inability to use cross-references.

[sortrefer-frank]

Would it be possible to prioritise implementation of references to DataTypes from another specification within Apicurio (rather than external references initially)?

The reason I say this is that some cross-reference functionality is necessary for any non-trivial usage of Swagger. For example, we are implementing various services that share data types (e.g. addresses, people etc.) and would like to use Apicurio for the specifications for all of these. However, at the moment we cannot because of the inability to use cross-references.

Agreed, this would bring us into killer app territory.

[EricWittmann]

@jsenko It looks like once we wrap up the registry impl, we have a clear priority from the community. :)

[Seeknode]

i'd love to see this, using apicurio at work and references would be incredibly useful to different teams.

[EricWittmann]

I realize this has taken a little while, but external references will be officially supported as of Beta 2.46, which is the next release (will hopefully be done tomorrow). There is a new UI to import Data Types and Responses from other Apicurio Studio documents into the current one. In addition, validation has been updated to support resolving external http/s as well as Apicurio-internal references. This should be a good first step towards mature support for references across documents.