Validation using JSON schema

[joshuatf]

Within various places around Woo, we have more and more cases where client-side validation proves beneficial.

Unfortunately, we don't have any real patterns for this so far and we seem to be taking a multitude of approaches to try and solve the same problem.

Current validation patterns

Typically our approaches when building out more modern user interfaces falls into one of two buckets; both of which have their pitfalls.

Custom client-side validation

The new product editor contains many fields which must be validated prior to saving products. Many of these fields implement custom blocks for the sake of being able to add their own validation logic within the block. Others are able to circumvent the need for custom blocks by using helper properties that handle validation within the templating API.

While this does seemingly offer a better experience, it can easily lead to the server and client becoming out of sync when it comes to those validation rules. As seen in the product editor, existing records have the potential to be completely valid in the previous experience, but can't be saved in the new experience given the introduction of client-side validation. (see pcQJnU-2nH-p2)

Server-side only validation

As an alternative to this, we can keep the single source of truth on the server and do validation there. The checkout block is extensible with additional fields from third parties. Currently, validation for these fields are handled on the server.

While this clearly leads to less errors and is a more stable approach, it also requires more round trips to the server to show validation errors and ultimately creates a much slower user experience for customers at a time when speed and reduction in friction is most important.

Why use JSON schema

Using JSON schema allows us to declare validation rules and properties to cover a large number of our validation cases. It is well-documented and allows us to easily extend our current schemas to support the necessary validation. But beyond that, it offers a solution to the bigger issues we're facing with validation right now:

1. Single source of truth

The biggest reason to use JSON schema is it gives us a sane way of understanding validation logic across both the client and server. This means that because they are using the same ruleset, there's little room for validation to become out of sync where server validation might pass, but client-side fails.

It also provides a self-documenting object that describes how data should be presented and allows easy reuse of validation rules going forward in different contexts.

2. Improved user experience

Because the validation can easily be used in the client, we don't need to call the server to check if a field is valid in many cases and can show errors in realtime as they are typed.

3. Easier third party integration

It allows for 3PDs to add validation logic in PHP via hooks that already exist. This reduces the need to spin up custom components or blocks in order to write validation logic in Javascript when generic block types already exist to extend existing experiences.

Proof of Concept

I put together a quick concept that shows how we can use JSON schema. The concept is pretty simple and piggybacks on our existing product endpoint schema. In brief:

1. We start by adding any validation rules to our existing schema. Third parties can and should extend this schema with their own properties and validation rules. This can already be achieved through existing hooks.

public function get_item_schema() {
	$schema = array(
		'$schema'    => 'http://json-schema.org/draft-04/schema#',
		'title'      => $this->post_type,
		'type'       => 'object',
		'properties' => array(
			...
			'stock_quantity' => array(
				'description' => __( 'Stock quantity.', 'woocommerce' ),
				'type'        => 'integer',
				'context'     => array( 'view', 'edit' ),
				'minimum'     => 0,
			),
			...
		),
	);

	return $this->add_additional_fields_schema( $schema );
}

2. We then pass the schema and record to an HOC that automatically handles validation and error creation based on the provided rules.

const [ productSchema, setProductSchema ] = useState< OptionsSchema | null >( null );

useEffect( () => {
	// Note that we could hydrate the page with the schema or preload this endpoint in some scenarios to improve page load times.
	apiFetch< OptionsResponse >( {
		path: 'wc/v3/products',
		method: 'OPTIONS',
	} ).then( ( result ) => setProductSchema( results.schema ) );
}, [] );

const { editedRecord } = useEntityRecord< Product >(
	'postType',
	'product',
	productId
);

<ValidationProvider
    schema={ productSchema }
    record={ editedProduct }
>
    { children }
</ValidationProvider>

3. In order to then use those validation errors somewhere, we simple grab the errors from the validation context, which contain everything from the error type, to the property, to the specific error message:

const { errors } = useContext( ValidationContext );

4. It is then up to the consumer to decide how to include the errors in components. Errors can exist in the initial state and the consumer decides whether or not to display them or wait until a field is touched/submitted before displaying them.

const [ isTouched, setIsTouched ] = useState( false );

<TextareaControl
	help={ isTouched && errors?.custom_field?.message }
	onBlur={ () => setIsTouched( true ) }
/>
) }

Issues that need more discussion

Despite all the positives that come with JSON schema, it is not perfect and there are areas that need more thought/discussion.

Complex validation use cases

While JSON Schema has come a long way and probably meets the needs of 95%+ of cases, there are some fields that are too complex to be handled purely in JSON or break the schema.

A field wanting to set a minimum amount based on another field can be handled by JSON schema, but a field wanting to set a limit based on a percentage of another field may be significantly more challenging to handle via pure JSON.

There are also cases that while possible to achieve, would be needlessly complex via JSON alone and validation on the server makes a lot more sense. For example, validating that a slug is unique is something we still want the server to handle as opposed to providing an irrationally long list of exclusions via JSON schema.

Related to the point below, but it is possible to declare custom keywords that may be able to handle this, but we would need to push towards having consistent dialect on both the client and server. Maintaining an organized package with all the validation logic would go a long way in making sure we can accomplish this.

Coerced types or non-conforming specs

Some fields, such as regular_price, would benefit from being a number type to allow usage of all the validation rules around numbers. However, WooCommerce treats prices as strings, meaning rules like exclusiveMinimum do not apply to this data type.

We can either set this type to number and attempt to coerce the string to a number at runtime during validation or we can create rules that aren't in the spec, such as exclusiveMinimumCoerced that allow keeping the type untouched while declaring a new rule to apply to this type.

On a side note, the Ajv package does offer an option to coerce data types but it's buggy when it comes to referencing other fields within the schema.

Error message consistency

The POC shows a way to provide error messages per each scenario as part of the schema. However, realistically we often have differing language when it comes to error messages on the client and server.

We need to pick between either providing consistent error messages or providing another API to intercept errors and provide messages based on error type and context.

Bundle size

Finally, it's also worth noting that package used for the client-side validation, Ajv, comes in at 119kb minified and 35kb when gzipped. While not extremely large, this is a non-insignificant amount. If we plan to use this on the frontend of the site, it may be worthwhile to consider creating a package that's a bit more stripped down and lighter weight to accomplish what we need.

Thoughts or concerns

Personally, I feel it's a pretty easy call to green-light this project since the benefits far outweigh the drawbacks. The only concerns I have currently are listed above and mostly just discussions that need decisions, not blockers.

But all feedback is welcome on this and if there are cases where this does not adequately fit project needs, I'd love to know what those are. Obviously JSON schema will not solve 100% of use cases, but if it can solve the overwhelming majority of cases, we should consider this approach to validation.

[mdperez86]

I like this approach to cover the validations and I do agree that this is not going to cover all possible use cases that we can have, but in my opinion, is a good way to follow. JSON schema has been widely used in micro service architectures as constraints/validations between API contracts. Also in some cases to build dynamic ui pages and forms.

One of the areas I faced and found not trivial to achieve is nesting schemas that are defined in different endpoints.
Lets say that there is a Product schema that contains a list of Attributes and both schemas belong to different APIs:

• In this case the schemas need to be loaded separately while building the ui page because they are not defined in one place but the validation manager should triggers the validations in both schemas in the same click event (well this always depends on the UI implementation) but you got my point.
• On the other hand schema definitions cannot handle complex and/or async validations like the slug one you just mentioned, but they can tell consumers how to do it, like hitting a specific endpoint that asynchronously manage the validation.
• When use it only for validations purposes, then how to manage the validations when the UI is presenting less fields than the ones defines in the schema? Consider the case where the schema validates the price field, but the UI does not render it.
• What about the error messages? Are them going to be translated server side and be the same in all contexts? Our experience tell us that error messages are always custom, even when they belongs to the same field, because the context/story telling can affect it.
• What happens when we have different templates representing different products with different validations, only one product schema will not scale to cover them all. Should the schema be defined in the product endpoint or somewhere else? Should it be just one definition or many? Should the schema be tied to the product and/or to the template?
• How can we link the data validation to the specific field that needs to be highlighted/focused? Consider that we need a reference to the field to focus it when the validation was triggered. Also how to keep the visual order so that the first visible field receives the focus.
• How all of this can be connected with the block system we have currently follow in the product editor?

Well these are some of my concerns about this approach (which I'm not seeking for answers) that of course they can be implemented, but I'm just put them on top of the table to be considered as part of the overall architecture of this solution.

[joshuatf]

Great thoughts all the way around, @mdperez86! Really appreciate you taking the time to leave this feedback.

In this case the schemas need to be loaded separately while building the ui page because they are not defined in one place but the validation manager should triggers the validations in both schemas in the same click event (well this always depends on the UI implementation) but you got my point.

Good point. This probably requires further investigation, but I wonder if we create a filter to autoload referenced schemas so when we retrieve, we have all the rules.

but they can tell consumers how to do it, like hitting a specific endpoint that asynchronously manage the validation.

💯 Creating definable passthrough endpoints that can be called and consistent responses will be key in doing this.

When use it only for validations purposes, then how to manage the validations when the UI is presenting less fields than the ones defines in the schema? Consider the case where the schema validates the price field, but the UI does not render it.

Ideally, the validation and the rendered fields should not be tied together. When doing validation we should always validate against the entire record and a record should always be valid regardless of what fields are shown.

If a field does not need to be entered when another is being used, the validation should represent this as "either property A or property B should be present." Writing this in JSON schema is arguably more complex than it needs to be, but we can discuss whether or not we add helper functions to assist in forming this JSON.

What about the error messages? Are them going to be translated server side and be the same in all contexts? Our experience tell us that error messages are always custom, even when they belongs to the same field, because the context/story telling can affect it.

Agreed on this point. We might not end up relying on the error messages sent from the server and instead rely on the error object that contains information about the error and offending keyword. This should be pretty trivial to do, but we'll of course need to make this extensible in the client if we choose to create custom errors there.

What happens when we have different templates representing different products with different validations, only one product schema will not scale to cover them all. Should the schema be defined in the product endpoint or somewhere else? Should it be just one definition or many? Should the schema be tied to the product and/or to the template?

Related to the point a few comments up, but the validation should happen at the record level and not the UI. It's the UIs responsibility to show those errors, but a record should always be valid and follow the same validation rules regardless of what template is being shown.

How can we link the data validation to the specific field that needs to be highlighted/focused? Consider that we need a reference to the field to focus it when the validation was triggered. Also how to keep the visual order so that the first visible field receives the focus.

Good question. I think we can solve this in a couple ways.

1. We can keep the errors in array format and rely on the first index allowing the component to handle grabbing focus:

useEffect( () => {
  if ( errors[0].this_field && formIsSubmitted ) {
    thisFieldRef.focus();
  }
}, [ errors, formIsSubmitted ] );

This of course won't focus a field if the first error isn't represented in the UI but that's a separate issue and we'll need to represent these errors somewhere either way.

2. If always focusing is a priority then we can register refs for errors and create a hook to handle this for us. It might make sense to keep this separate from the actual validation hooks.

function MyField() {
  useAutoFocusOnError( propertyToWatch, thisFieldRef );
}

How all of this can be connected with the block system we have currently follow in the product editor?

I think a lot of it we can rely on block context, but hooks are also an option. We'll probably need to evaluate this on a case by case basis.

[retrofox]

When use it only for validations purposes, then how to manage the validations when the UI is presenting less fields than the ones defines in the schema? Consider the case where the schema validates the price field, but the UI does not render it.

How can we link the data validation to the specific field that needs to be highlighted/focused? Consider that we need a reference to the field to focus it when the validation was triggered. Also how to keep the visual order so that the first visible field receives the focus.

These points are indeed valid, but they are not part of the validation process per se. It's essential to isolate this process from the UI entirely. Currently, it's mandatory to render all fields blocks, even those that are not visible, because of the form validation. The blocks should consume the Validation API instead of handling it.

With a Validation API, independent of the view, we can render errors and messages wherever we want. For instance, we can show a validation sidebar when the form contains errors, right after the user tries to save the post.

[senadir]

Another option would be to build something similar or reuse the Laravel validation system, which I found to be more approachable than a JSON schema

https://laravel.com/docs/11.x/validation#available-validation-rules

For Checkout additional fields, we're looking into adding visibility rules, (doc here). Ideally if we're going to come with a validation system, it should look slightly similar to that one and ideally can both be used together (under the same definition).

[joshuatf]

Nice! I like the idea of something like Laravel's validation system if it's acting as an ORM to generate JSON rules.

Correct me if I'm wrong, but Laravel's validation system does not generate any JSON and is only available server-side, right? We do need a reliable way to validate client-side as well, but I think there's a place for a Laravel-like system to help generate more complex JSON schema structure.

[senadir]

Correct me if I'm wrong, but Laravel's validation system does not generate any JSON and is only available server-side, right?

The basic one doesn't no, but it can be enabled via Precognition library: https://laravel.com/docs/11.x/precognition thou it seems to do that by doing automatic requests behind the scenes, so it's not a valid option to generate a JSON.

[senadir]

One main concern I have with JSON is the complexity of the system, and lack of ability to get dynamic-ish values. Store API Schema for example has a good amount of boilerplate in it, would it be possible that developers write a simpler version of the rules that gets translated down the road to JSON?

[joshuatf]

lack of ability to get dynamic-ish values

Do you have some examples of this we could use as a test case?

would it be possible that developers write a simpler version of the rules that gets translated down the road to JSON?

Mentioned in your above comment as well, but I definitely think there's some room to make writing complex validation rules easier. Could you provide an example here as well of something that feels complex to write in JSON schema so we can use that for the sake of discussion?

[senadir]

Do you have some examples of this we could use as a test case?

This is only valid if we're going to write straight out .json files, not a PHP one that generates the JSON.

When it comes to dynamic values, one thing to consider is the context those validation rules will run into, should they be expected to access existing fields, will they need to access merchant session or not.

This is a concern for us in Checkout and something we're still wondering about because we want to run validation in some places, like the Checkout itself, but also the my account screen, the order management screen, and possibly the mobile app. This is an issue we're currently facing with visiblity rules.

[senadir]

Could you provide an example here as well of something that feels complex to write in JSON schema so we can use that for the sake of discussion?

I find the schema needed to accept a field in REST API to be too complex and boilerplaty, an example:
https://github.com/woocommerce/wceu23-shipping-workshop-final/blob/main/shipping-workshop-extend-store-endpoint.php#L55-L82

[joshuatf]

@senadir I suppose those are a little boilerplatey, but I do see most of the options defined as necessary. Do you think it would be better if we had tools that handled most of the defaults? Or did you have something else in mind?

$schema[ 'myStringField' ] = SchemaTool::string( 'My description' );

[dmsnell]

Sharing some thoughts I had previously shared privately.

---

While JSON Schema does provide some level of validation, the schemas can get complicated to express and then become bloated themselves.

Depending on what you do, it might be worth checking into Zod, which @sirreal introduced into some other projects.

Zod won’t validate the types on the server, but it does provide an arguably more convenient way to express the types of data (because it’s composable and code-oriented) while providing auto-completion inside the JS/TS code. Once you parse a document with what Zod auto-generates, it’s guaranteed to be valid, vs. JSON Schema where it can be hard to properly validate certain constraints.

[mattsherman]

what are the (tangible) additional benefits that Zod provides us over generating TS type declarations if we do have other schema generation tools on the server

TypeScript only provides static type checking at compile time.

Zod, and similar libraries, provide dynamic runtime validation.

I don't see how TypeScript types alone could be used for the sort of validation we are talking about. How would we ensure, at runtime, that a number fit within a min and max, for example?

[dmsnell]

In general I'll caution projects wanting to generate types from PHP classes, reflection, or static analysis. This isn't a reason not to do it, but it can happen that those attempts are based on properly understanding the PHP and then enforcing it in JS. The problem here is that PHP often gets used in ways we don't expect, and our types misalign.

In existing applications we can accidentally break the application by enforcing insufficient types. Consider one extremely common example with WordPress projects: a number appears as a string of digits. Since PHP doesn't generally enforce its own types, things can happen in disconnected places we might overlook. Moral of the story is to be aware of this if attempting to do the complicated process of auto-generating types from PHP and figure out a plan to relax the type errors in the client (something I think @sirreal's work did when it was introduced).

I've seen the same issue where turning on declare(strict_types=1) on existing PHP code breaks the project because the types were unreliable hints.

One can imagine generating these type definitions and then applying them on egress from the server before relying on them in the client. Any errors can be reported and alerted until they are reliable enough to enforce.

---

Also one final note, which I'll never pass the chance to reiterate: types are not inherent to a data structure. Types are constraints that we apply for a purpose. For the raw purpose of avoiding a crash in the client this can be fine to auto-generate types, but we will miss the power that types offer us if we delegate the design work to the bot.

Types, like functions and interfaces, communicate something about the data. Is that number an id? an age? a quantity?

One further approach here is to start by figuring out what everyone wants the types to communicate and then build types first, applying them down to JS and PHP whereby both are secondary to the definitions. This is a long approach and doesn't help with immediate crashes, but it can be a valuable way to improve both code bases over time and communicate more clearly to third-party devs. That is, instead of describing the data we have, describe the data we expect and then make sure the data we have matches that.

[joshuatf]

@mattsherman I'm running under the assumption that we already have a validation package of our own in place given it's going to need to be lightweight and capable of being bundled with frontend scripts.

I can see the argument for Zod if we want to use it solely in place of those validation scripts. It is smaller than some of the alternatives when minified and gzipped, but I'm also confident we could come up with a lighter weight solution with only the essentials if we choose to make that investment.

[reykjalin]

As someone with less first-hand knowledge around Zod, what are the (tangible) additional benefits that Zod provides us over generating TS type declarations if we do have other schema generation tools on the server? My gut feeling is leaning towards the TS type generation without Zod, but there may be other benefits that come with Zod that make bundling it worthwhile.

Like @mattsherman mentioned; Zod provides runtime guarantees that the data you're parsing is actually what you expect it to be, whereas TS types only help at compile time, which may not be correct in practice. Consider, for example, someone calling APIs from JS where they don't have the TS type guarantees. Zod will work regardless of whether the caller is using JS or TS (or ReScript or Fable or …).

In short: with TS you make assumptions and act based on those assumptions, with Zod you know exactly what you're working with.

There are other validations zod can make that are stricter than what TS can provide. As an example, you can verify that a string is a valid datetime string, something I don't think TS can do natively with it's type system. You can also do various number validations at runtime. And there are even more options zod can provide.

In general I'll caution projects wanting to generate types from PHP classes, reflection, or static analysis.

I think there's a time and a place for doing this. For example I wouldn't advise generating types based on something returned by a service class, or when a REST controller bundles together different data structures from different parts of the codebase. But I do think it's valid to do this for model classes, when those models are sent "as-is" to the frontend.

Even then, to do this properly you'd probably want to use something like attributes to add metadata for additional validations not covered by TS's or PHP's type systems. This would be a significant investment that may or may not pay off down the line. Especially when using 3rd party libraries.

But I find the idea interesting enough to at least explore, and I think there's a place for the approach even if it's strictly applied to model classes.

The problem here is that PHP often gets used in ways we don't expect, and our types misalign.

I actually see this as a reason to verify our assumptions in the front-end, regardless of whether or not the types/zod parsers are generated based on the PHP code or not 🙂

[joshuatf]

In short: with TS you make assumptions and act based on those assumptions, with Zod you know exactly what you're working with.

I see, thanks for the explanation @reykjalin!

We do have a lot of tooling in place (at least in Woo) that handles coercion to specific types (currencies, dates, etc), but this largely manual and I can see some advantages to using Zod in place of some of these tools.

I guess the 2 underlying questions are:

1. Do we try and come up with a creative solution mentioned above where Zod gets parsed to a schema and copied to the server (or the server schema dictates Zod in some way). Ideally the server remains the source of truth.
2. Whether or not this library is light enough to be considered for use on the frontend of the site versus creating our own.

I actually see this as a reason to verify our assumptions in the front-end, regardless of whether or not the types/zod parsers are generated based on the PHP code or not 🙂

➕ 1️⃣