Take 2: create model from JSON schema aka model_load_json_schema()

[fabiog1901]

Hello, with the V2 release I'd like to take the opportunity and dare to ask the eng team on a topic that was raised before for which only workarounds or very bespoke solutions where proposed.

In short, there is a method for dumping the Model to JSON, model_json_schema but there is no counterpart, that is, a method that [re-]loads from JSON to a Model.

The use case is simple: I want to give my users the capability to store arbitrary artifacts. An artifact can be anything, any object: a car, a travel trip, a database cluster, a toilet, you name it. But it has to conform to a predefined schema for that object.

So an admin user would say, fine, we create the model schema (using the pydantic syntax). Then users create objects, making sure they conform with the model schema. This validation/sanitization involves loading the model from the saved json schema representation, and checking it against the user's input json payload.

Example model:

class Car(BaseModel):
    id: int
    name: str | None = Field(max_length=50)

I'd save its JSON schema, below, in the database for long term storage.

{
    "properties": {
        "id": {
            "title": "Id",
            "type": "integer"
        },
        "name": {
            "anyOf": [
                {
                    "maxLength": 50,
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Name"
        }
    },
    "required": [
        "id",
        "name"
    ],
    "title": "Car",
    "type": "object"
}

Then, when a user wants to create a car and passes a json payload (a dict) such as {'id': 123, 'name': 'SuperCar', 'didntask': 'whatever'}, I'd load the Pydantic Model dynamically using create_model, or a hypothetical model_load_json_schema(), feeding the above JSON schema, and validate/sanitize the user's input:

# Car = create_model("Car", BaseModel, the_json_string)
Car = model_load_json_schema(the_json_string)

# validate/sanitize - i.e. conform to the schema
car = Car(**{'id': 123, 'name': 'SuperCar', 'didntask': 'whatever'})

print(car)
'''
id=123 name='SuperCar'
'''

Then I can safely store the car.model_dump_json() output to the database.

[mohandoz]

I'm actually working on this, i want the user to define his dynamic schema configuration from the UI and save them in DB to accept Dynamic api calls validated using the defined schema, still looking for a solution

[samuelcolvin]

Sure datamodel-code-generator is what you looking for https://docs.pydantic.dev/latest/integrations/datamodel_code_generator/.

Or you could write something that converted JSON Schema to pydantic core schema directly.

We won't be adding this functionality directly to pydantic.

[mohandoz]

Just stumbled on the projects minutes ago Thanks!!

[h55nick]

@mohandoz @fabiog1901 I'm trying to do the same thing and datamodel_code_generator isn't going to be a solution for me.

This is where I'm at:
https://gist.github.com/h55nick/0e1d48b72ab9c5db33b89ba0443f41e7
but there are some issues with this current approach.

Would love to collaborate.

@samuelcolvin thanks for your work on Pydantic.