Feature: formating of errormessages for display to end user

[cork]

I'm trying to figure out how jsonschema.ValidationError are to be translated to something understandable to an end user, or for that matter looped to for example mark input fields for the error.

I get string like "additioalProperties 'test1', 'test2', 'test3' is not allowed" or "missing properties: 'firstName'". and no separate field to check the list of the fields with the error or what a machine understandable version of the error is.

Am I missing something or isn't there a way to do anything except showing the result as a string blob?

[hbagdi]

I'm trying to do something similar.
In my case, I'd like to introspect the error and get a reference to the Schema node so that I can populate a helpful error for the end user.

So when I get something like:

[I#] [S#] doesn't validate with file:///tmp/schema.json#
  [I#/connect_timeout] [S#/properties/connect_timeout/minimum] must be >= 1 but found -1
  [I#] [S#/allOf/0] allOf failed
    [I#] [S#/allOf/0/then] if-then failed
      [I#/protocol] [S#/allOf/0/then/properties/protocol/const] value must be "https"

I ultimately want to access the "title" or "description" field of the if or else and return that to the user.

Is there a way to fetch the Schema based on the KeywordLocation of an error?

[santhosh-tekuri]

@cork api to access values from error message could be potential feature, may be for next release

@hbagdi you can navigate *Schema using KeywordLocation as if it is simple xpath. but this might be tricky if the schema uses $dynamicRef

[hbagdi]

@hbagdi you can navigate *Schema using KeywordLocation as if it is simple xpath. but this might be tricky if the schema uses $dynamicRef

Are there any tools or helper functions to do this? I looked into the code but didn't find anything close.

I see you've https://github.com/santhosh-tekuri/xpath but I don't have a remote idea on how that can parse a JSON Schema. Any pointers to dig deeper?

[santhosh-tekuri]

sorry. not xpath. as jsonpath
for example consider keywordLocation /allOf/0/then/properties/protocol/const

you split it by / and for each part navigate struct

so golang navigation for keywordLocation /allOf/0/then/properties/protocol/const is

schema.AllOf[0].Then.Properties["protocol"].Const

[santhosh-tekuri]

you can find some sample code here

the sample code works on json objects like map/interface.

in your case you have to do it for *Schema struct

[hbagdi]

@santhosh-tekuri Thank you for your response here. Your commentary was very helpful for me to arrive at a solution.

[Surfoo]

Hello @santhosh-tekuri,

What do you think to implement only one string per error and custom error with templating like https://github.com/xeipuuv/gojsonschema? (This lib seems unmaintened...) It will be a great feature 👍

[TeCHiScy]

api to access values from error message could be potential feature, may be for next release

hi @santhosh-tekuri, thanks for your great work first. I'd like to vote on the api, with an api to get the keywordPath and the args in the validationError, it will be very easy for developers to format validation errors into what they want.

[Oct17th]

hi @santhosh-tekuri，I have a similar situation here，I hope there has a interface that i can define custom error template，did you have plan to support it？

[santhosh-tekuri]

Started working on this feature. Will post pull request tomarrow

[santhosh-tekuri]

please check pull request #99
this pull request changes type of ValidationError.Message from string to struct implementing fmt.Stringer

go through the changes and let me know your opinion

[santhosh-tekuri]

fix available in v6.0.0-alpha.1

[AustinAbro321]

Hi everyone, I too had a similar case where I didn't want to use https://github.com/xeipuuv/gojsonschema because it seemed unmaintained but I liked the more actionable format for users. I ended up implementing a way to get a similar format to what https://github.com/xeipuuv/gojsonschema does but with this package. The below function returns error strings that look like the below block. In this case the schema had two things wrong with it and we got the following two entries in the returned array. @santhosh-tekuri If you'd like I can make a PR adding this function to be built into the package, I think many would find it useful.

- /components/0/import: additionalProperties 'not-path' not allowed
- /components/1/import/path: expected string, but got number

func getChildCauses(validationErr *jsonschema.ValidationError, childCauses []error) []error {
	if validationErr == nil {
		return childCauses
	}
	if validationErr.Causes == nil {
		errMessage := fmt.Errorf(" - %s: %s", validationErr.InstanceLocation, validationErr.Message)
		return append(childCauses, errMessage)
	}

	for _, subCause := range validationErr.Causes {
		childCauses = getChildCauses(subCause, childCauses)
	}
	return childCauses
}

[erezrokah]

Hi 👋 Thank you for this lovely package, dropping an example from a JavaScript package (AJV) for https://ajv.js.org/packages/ajv-errors.html#messages-for-properties-and-items. I know that it's not "proper" JSON schema, but I think that in some cases it can be quite hard for the underlying validation library to provide a user friendly error message.

A common example is using patterns to validate strings. The underlying library can mostly error out with value does not match pattern <x> or something similar. But the author of the pattern, understanding the context of it, can for example configure an error as value must contain at least one template variable in the format {{var}} (let's say our pattern is ^.*{{.+?}}.*$).

Other examples can be usage of if,then,anyOf or any kind of schema composition, where you might want to highlight a specific failure.

[andrewpollock]

Hello, I'm exploring JSON Schema validation (in particular the UX of it) at the moment.

I started with https://github.com/xeipuuv/gojsonschema and like others, was concerned about its apparent lack of maintenance, and have arrived here. The CLI validation output is nice, but can still be rather impenetrable for mere mortals.

I'm validating the OSV Schema:

$ ~/go/bin/jv ~/gosst/osv/osv-schema/validation/schema.json ~/gosst/osv/osv-validation/lastaffected_and_fixed-CVE-2023-41045.json 
[I#] [S#] doesn't validate with https://raw.githubusercontent.com/ossf/osv-schema/main/validation/schema.json#
  [I#/affected/0/ranges/0] [S#/properties/affected/items/properties/ranges/items/allOf/1] allOf failed
    [I#/affected/0/ranges/0] [S#/properties/affected/items/properties/ranges/items/allOf/1/then] if-then failed
      [I#/affected/0/ranges/0] [S#/properties/affected/items/properties/ranges/items/allOf/1/then/not] not failed

I had high hopes of being able to judiciously insert some title and/or description fields into more of the sub-schemas, but so far that doesn't seem to have been fruitful.

[santhosh-tekuri]

simple titleor description cannot explain various ways a schema validation can fail. so the error messages cannot be replaced with them generally. For you use case it may be appropriate.

since you can get schemaLocation for validationError, from schemaLocation you can extract your custom error message.

[andrewpollock]

since you can get schemaLocation for validationError, from schemaLocation you can extract your custom error message.

My feedback was more on the jv tool as much as the library behaviour. I'm exploring validation tooling for producers of OSV records.

From further research:

Second, collecting annotations is also a recommendation, not a requirement. Sadly, few implementations actually support it.

So it sounds like there's an opportunity to be utilizing title and description?

I played around with andrewpollock/osv-schema@a923795 and andrewpollock/osv-schema@cf4d326 but the addition of title had no impact on the user-visible output.

[santhosh-tekuri]

what I meant is title and description are not meant for showing error messages. they are like comments for schema authors to explain what schema does.

if you want to use them for errors, you may have to modify the tool jv accordingly.