Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiler short circuits when validating schemas, not providing the full list of errors #116

Open
hysmio opened this issue Apr 28, 2023 · 0 comments

Comments

@hysmio
Copy link

hysmio commented Apr 28, 2023

Hey! Noticed something while building while using extensions. It seems that Compile short circuits validation when errors occur in a document. I imagined that it would validate all schemas even if errors occurred in one.

This very well could be a design decision, but wanted to clarify.

4 fairly verbose examples can be found here:
https://goplay.tools/snippet/dhM5tkI82j1

The examples are:

  1. returns meta missing properties

Actual output is the expected output

jsonschema file:///home/user/projects/golang-playground/schema.json compilation failed
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json#
  [I#] [S#/required] missing properties: 'foo', 'foobar'
  1. returns https://json-schema.org/draft/2020-12/schema validation error, but not meta extension validation error

Actual output:

jsonschema file:///home/user/projects/golang-playground/schema2.json compilation failed
[I#] [S#] doesn't validate with https://json-schema.org/draft/2020-12/schema#
  [I#] [S#/allOf/3] allOf failed
    [I#] [S#/allOf/3/$ref] doesn't validate with 'https://json-schema.org/draft/2020-12/meta/validation#'
      [I#/minLength] [S#/properties/minLength/$ref] doesn't validate with '/$defs/nonNegativeIntegerDefault0'
        [I#/minLength] [S#/$defs/nonNegativeIntegerDefault0/$ref] doesn't validate with '/$defs/nonNegativeInteger'
          [I#/minLength] [S#/$defs/nonNegativeInteger/minimum] must be >= 0 but found -1

Expected output:

jsonschema file:///home/user/projects/golang-playground/schema2.json compilation failed
[I#] [S#] doesn't validate with https://json-schema.org/draft/2020-12/schema#
  [I#] [S#/allOf/3] allOf failed
    [I#] [S#/allOf/3/$ref] doesn't validate with 'https://json-schema.org/draft/2020-12/meta/validation#'
      [I#/minLength] [S#/properties/minLength/$ref] doesn't validate with '/$defs/nonNegativeIntegerDefault0'
        [I#/minLength] [S#/$defs/nonNegativeIntegerDefault0/$ref] doesn't validate with '/$defs/nonNegativeInteger'
          [I#/minLength] [S#/$defs/nonNegativeInteger/minimum] must be >= 0 but found -1
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json#
  [I#] [S#/required] missing properties: 'foo', 'foobar'
  1. returns meta2 required properties error, but not meta#/foobar/minLength

Actual output: (this shows two children errors in one message)

jsonschema file:///home/user/projects/golang-playground/schema3.json compilation failed
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json#
  [I#] [S#/required] missing properties: 'foo'
  [I#/foobar] [S#/properties/foobar/minLength] length must be >= 3, but got 1

Expected output:

jsonschema file:///home/user/projects/golang-playground/schema3.json compilation failed
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json#
  [I#] [S#/required] missing properties: 'foo'
  [I#/foobar] [S#/properties/foobar/minLength] length must be >= 3, but got 1
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta_2.json#
  [I#] [S#/required] missing properties: 'bar'
  1. **Validate returns the same singular meta error when calling .Error() but

DetailedOutput() returned the correct errors**
Actual output:

jsonschema: '' does not validate with file:///home/hayden/projects/golang-playground/schema4.json#/maximum: must be <= 8 but found 9

Expected output: (ValidationError.GoString())

[I#] [S#] doesn't validate with file:///home/hayden/projects/golang-playground/schema4.json#
  [I#] [S#/maximum] must be <= 8 but found 9
  [I#] [S#/foobar] when foo is bar, numbers must be >= 10, but got 9

I've created a PR here for Compile supporting multiple errors.

For ValidationError.Error() I would suggest using GoString or creating a way to show multiple errors.

@hysmio hysmio changed the title ValidationError only contains errors from the first document it validates against Compiler short circuits when validating schemas, not providing the full list of errors Apr 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant