Support for more recent JSON Schema versions

[gregsdennis]

Currently this project supports up to draft 7, which was released March 2018. Since then, two new versions have been release: 2019-09 (which probably should be skipped) and 2020-12 (which definitely should be supported).

What help would be needed to update this project to support the latest versions of JSON Schema?

[dblanchette]

Hi!

I'm sorry for not maintaining and evolving the project much recently.

I read the changelogs for 2019-09 and 2020-12 and I think there are some quick wins.

The idea would be to support the old and new keywords interchangeably, even if that means accepting schemas that are not well structured or are using keywords that are not available in the version declared in the schema.

I also don't know if the schemas that are found in the wild do declare their draft versions explicitly, because that would help tremendously. In that case, there could just be different code paths depending on the draft version.

Here are my exploratory notes.

Feel free to submit PRs or comment with your ideas!

2019-09

(This is not a complete list)

$anchor

When recording and resolving references in intermediate_representation.py, the anchor should be taken into consideration.

$defs

Really easy as it behaves exactly like definitions

$recursiveAnchor and $recursiveRef

I read that part several times and I'm stil not sure how it works exactly :\

$ref

The changelog states Other keywords are now allowed alongside of it. This is already accepted, so nothing to do here

$vocabulary

This is only useful for metra-schemas, so nothing to do here (I think)

dependentSchemas and dependentRequired

dependencies is not currently supported, so this is a bonus.

unevaluatedItems, unevaluatedProperties, maxContains, and minContains

Those can be added simply by modifying the templates and adding an explanation

deprecated

There is a support for this by looking up in the description for [Deprecated, so the change would be simply to also check if the keyword is there.

2020-12

prefixItems and items

Now items has two meaning depending on the draft version. If the version is declared, then no problems, if it is not, we can manage since we know that if items is an aray it is the old version (array of schemas to be applied like a tuple) and if it is not an array, then it is the new kind (schema for all items)

$dynamicRef and $dynamicAnchor

Still not sure I understand completly... I need to read more on this

contains and unevaluatedItems

Currently, contains is documented as follow: At least one of the items must be:. That wording is still applicable. For, unevaluatedItems, the wording could be something like Items that do not match other constraints should be:

[rogueturnip]

Thumbs up on this work. I'm using the $refs, $defs, contains and items values from 2020-12 and it would be really good to have those supported.

[gregsdennis]

I also don't know if the schemas that are found in the wild do declare their draft versions explicitly

Our experience is that it's hit or miss. It seems that most attempt to, but a lot that do don't do it right (either over-specify, like having $id: #/$defs/foo, or just get it wrong entirely).

---

Some additional notes:

First, I might not bother with 2019-09. We're trying to steer people away from it and just use 2020-12. Specifically the $recursive* keywords weren't specified well; the $dynamic* keywords do much better, and they're set to improve further in the next version.

$anchor

There was confusion around the behavior of $id when the value was a location-independent identifier, like #foo. This new keyword separates those two cases and becomes that value. So instead of $id: #foo, we now have $anchor: foo.

{$ref: #foo} will now resolve to the subschema that defines $anchor: foo.

$vocabulary

This is actually quite important. This keyword in a meta-schema defines which keywords can be used in a schema.

For example, if an otherwise known keyword, e.g. maximum is used in a schema whose meta-schema doesn't list the validation vocabulary, maximum should be ignored. There are tests around this in the test suite.

It's also the new extension point where you can define your own keywords. A prime example is my data vocabulary.

deprecated

This is an annotation-only keyword. It has no behavior.

unevaluatedItems and unevaluatedProperties

These work just like additional* except that they can "see inside" of sibling keywords like *Of and if/then/else. There are a bunch of tests for these.

As of 2020-12, contains is also considered for unevaluatedItems. In the next version, contains will also apply to objects, so additionalProperties and unevaluatedProperties will need to consider it as well.

minContains and maxContains

These behave as expected, mostly. A nuance is that when minContains: 0, contains always passes. Not especially useful, but a case we had to consider nonetheless.

$dynamicRef and $dynamicAnchor

I proposed this use case for the CloudEvents spec. These keywords can be used for more, but this gives a good idea of what they do.

There's also my blog post on using them to define schemas for generic types (which is basically the same as the thing for CloudEvents, just longer-winded).

[rogueturnip]

Could you do the version detection in 2 ways. If the $id is value process appropriately, if not fail out with an "invalid version" type message. In those cases an individual can then use a command line flag --schema or so and put in draft7, 2020-12 etc and it does the same thing. if the wrong schema is put in on the command line results will vary ;).

I feel if you are looking to generate using this tool you need to at least know what schema you are working against.