Entity Field Type Validation

[toBeOfUse]

Is your feature request related to a problem? Please describe.

Currently, if I have an entity that looks like this:

@Entity('tasks', { allowApiCrud: true })
export class Task {
  @Fields.autoIncrement()
  id = 0

  @Fields.string()
  title = ''

  @Fields.boolean()
  completed = false

  @Fields.string()
  shouldBeString = ''
}

and make a POST request to the API with a body like this:

{
    "title": "test insert",
    "completed": false,
    "shouldBeString": 89
}

the number 89 is successfully stored in the field that should be a string, at least with the default JSON file provider and the MongoDB data provider (and maybe others that I haven't tried.) This can wreak havoc if the object is then retrieved and something like toLowerCase() is called on the field that should be a string but is in fact a number. This means this code is vulnerable to anyone who knows how to manually make a POST request.

Describe the solution you'd like

I can work around this by creating new field types with built-in validation. This is my current working solution for strings:

function checkedString(options?: FieldOptions) {
  return Fields.string({
    ...options,
    validate(entity, fieldRef) {
      if (typeof fieldRef.value !== 'string') {
        throw fieldRef.metadata.key + ' should be a string'
      }
      if (options) {
        if (Array.isArray(options.validate)) {
          for (const v of options.validate) {
            v(entity, fieldRef)
          }
        } else if (options.validate) {
          options.validate(entity, fieldRef)
        }
      }
    }
  })
}

Could something like this be built into the field types? This would be helpful to other users, and I would not have to carry around these validated field types to other projects.

Describe alternatives you've considered

I tried inserting this invalid data with the SQLite data provider but it coerced the input values into the correct type as far as I can tell, so some (most?) databases will take care of this problem on their own. Still, it's dangerous for the JSON and MongoDB data providers and possibly others.

Additional context

I created a simple Github repo based on the Task MVC example that just adds a frontend that bypasses TypeScript type validation and lets you POST objects to Remult directly, like an antagonistic user could do with curl or from the browser console, displaying any errors that result. It also fetches the resulting database entries and displays them, so you can see if invalid data was added. This can be used to test field type validation in isolation.

[noam-honig]

@toBeOfUse, fixed it on https://github.com/remult/remult/releases/tag/v0.19.3

Care to check it out?

[toBeOfUse]

This fixes the problem; it even retroactively sanitizes the invalid values I'd been inserting upon retrieving them from the database. So it was just the string data type that was missing this coercion? This was an incredibly quick and accurate fix.

There is one more small problem that came up when I tested inserting objects with strings for fields that should be numbers through the API. If I have this entity:

@Entity('tasks', { allowApiCrud: true })
export class Task {
  @Fields.autoIncrement()
  id = 0

  @Fields.string()
  title = ''

  @Fields.boolean()
  completed = false

  @Fields.number({ allowNull: false })
  shouldBeNum: number = 5
}

And post this to the database:

{
    "title": "attempt to put string into number field",
    "completed": false,
    "shouldBeNum": "surprise, it is a string"
}

The resulting database entry is this:

{
    "id": null,
    "title": "attempt to put string into number field",
    "completed": false,
    "shouldBeNum": null
}

So, the invalid value for shouldBeNum is converted to a null, which is fine, but this bypasses both the default value given for that field in TypeScript and the allowNull option in the field options. I think this could still be a slight potential problem if someone assumes this field can never be null and calls, like, toFixed on it in some important code, and gets an error because a null was induced through invalid data being submitted through the API. But it's an edge case.

Also, when I try to fix this with Validators.required, Typescript complains:

It seems like here, the Validators.required validator is defined to only work on the valueType string:

remult/projects/core/src/validators.ts

Lines 5 to 13 in a2aeaad

	static required = Object.assign((entity: any, col: FieldRef<any, string>, message = undefined) => {
	if (!col.value || typeof (col.value) === "string" && col.value.trim().length == 0)
	col.error = message || Validators.required.defaultMessage;
	}, {
	withMessage: (message: string) => {
	return (entity: any, col: FieldRef<any, string>) => Validators.required(entity, col, message)
	},
	defaultMessage: 'Should not be empty'
	});

But it works fine; when using Validators.required, trying to add a field with a non-number value for a @Fields.number truly does not work. I think the declared types might be too strict though.

[noam-honig]

We've done considerable work on validations in the last few versions, including the coming exp version. Can you review it and see if it addresses your needs?

https://github.com/remult/remult/blob/master/CHANGELOG.md

https://remult.dev/docs/validation.html#validation