typing.Self not supported in generic fields

[strangemonad]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

It seems that with the recent introduction to support the Self type in pydantic 2.7 #5992, this doesn't cover support for generic fields where Self is the type var.

Alternatives:

1. For the code below, when making instantiate a new model, the FieldInfo likely needs to be updated. eg for the example below, Product has the following field info.

{'ref': FieldInfo(annotation=Ref[Self], required=True), 'name': FieldInfo(annotation=str, required=True)}

At the point of model creation, I believe it should be possible to update Ref[Self] to Ref[model_class]? The downside here is that the FieldInfo, while it reflects the reality of the semantics of the Self type, no longer matches the annotation verbatim making it harder to potentially debug, easier to introduce bugs (e.g. if the annotation value is used to look up the cached generic subclass) ad potentially incompatible with whatever direction the python core team takes __future__ annotations in.

2. Keep the existing field info and handle the case when building the field's core schema. This would mean a generic model subclass might need to be instantiated just in time and registered in then cache in the correct order for it to be visible to the core_schema machinery.

Example Code

from typing import TypeVar, Generic, Self
from pydantic import BaseModel

T = TypeVar("T", bound="Entity")


class Ref(BaseModel, Generic[T]):
    path: str


class Entity(BaseModel):
    ref: "Ref[Self]"


class Product(Entity):
    name: str

p = Product(name="test", ref=Ref[Product](path="f"))


```console
Traceback (most recent call last):
  File "/Users/shawn/Code/instance-bio/instance/apps/admin/src/pdg.py", line 18, in <module>
    p = Product(name="test", ref=Ref[Product](path="f"))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shawn/Code/instance-bio/instance/apps/admin/.venv/lib/python3.11/site-packages/pydantic/main.py", line 171, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Product
ref
  Input should be a valid dictionary or instance of Ref[Self] [type=model_type, input_value=Ref[Product](path='f'), input_type=Ref[Product]]
    For further information visit https://errors.pydantic.dev/2.6/v/model_type

### Python, Pydantic & OS Version

```Text
pydantic version: 2.7.1
        pydantic-core version: 2.18.2
          pydantic-core build: profile=release pgo=true
                 install path: /Users/shawn/Code/instance-bio/instance/libs/instance-firestore-entity/.venv/lib/python3.11/site-packages/pydantic
               python version: 3.11.4 (main, Aug 14 2023, 09:41:08) [Clang 14.0.3 (clang-1403.0.22.14.1)]
                     platform: macOS-14.4.1-arm64-arm-64bit
             related packages: typing_extensions-4.11.0 pyright-1.1.321
                       commit: unknown

[sydney-runkle]

@strangemonad,

Yep, you're right, this isn't currently supported. I've marked this as a feature request. PRs with support are welcome!

[NeevCohen]

@sydney-runkle Hey, I started looking into this, and it seems a little tricky to me.

In the definition of Entity

class Entity(BaseModel):
    ref: Ref[Self]

Ref's __class_getitem__ method gets called which is defined here, though when its called, the typevar_values parameter is just Self, without a way to access or determine what Self is actually referring to (none that I could tell at least).

My best idea was to perhaps look try to look up the stack to figure out what Self refers to, kind of like what super() does, though I'm not sure that's the best idea.

Do you perhaps have any ideas/directions into how this should be implemented?