Class properties have a potential for namespace collision

[jrkarnes]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

When you are using a forward reference (as a string literal to defer validation to Runtime), and:

• your class has a property that shares a name with another class AND
• you are using Optional[] = None for the definition of that property

Then pydantic can have a bit of a fit when it attempts to create an instance of the class.

Example Code

from pydantic import BaseModel
from typing import List, Optional

class RulesSource(BaseModel):
    StatelessRulesAndCustomActions: Optional["StatelessRulesAndCustomActions"] = None

class StatelessRulesAndCustomActions(BaseModel):
    StatelessRules: Optional[List["StatelessRule"]] = None

class StatelessRule(BaseModel):
    pass

def test_the_bug():
  # Create instances of the classes to trigger type checking
  rules_source = RulesSource(StatelessRulesAndCustomActions=StatelessRulesAndCustomActions(StatelessRules=[StatelessRule()]))

============================================================================================= FAILURES =============================================================================================
___________________________________________________________________________________________ test_the_bug ___________________________________________________________________________________________

    def test_the_bug():
      # Create instances of the classes to trigger type checking
>     rules_source = RulesSource(StatelessRulesAndCustomActions=StatelessRulesAndCustomActions(StatelessRules=[StatelessRule()]))
E     pydantic_core._pydantic_core.ValidationError: 1 validation error for RulesSource
E     StatelessRulesAndCustomActions
E       Input should be None [type=none_required, input_value=StatelessRulesAndCustomAc...Rules=[StatelessRule()]), input_type=StatelessRulesAndCustomActions]
E         For further information visit https://errors.pydantic.dev/2.7/v/none_required

Python, Pydantic & OS Version

% python -c "import pydantic.version; print(pydantic.version.version_info())"
             pydantic version: 2.7.1
        pydantic-core version: 2.18.2
          pydantic-core build: profile=release pgo=true
                 install path: <REDACTED>/.venv/lib/python3.12/site-packages/pydantic
               python version: 3.12.1 (main, Apr 29 2024, 09:58:38) [Clang 15.0.0 (clang-1500.3.9.4)]
                     platform: macOS-14.4.1-arm64-arm-64bit
             related packages: typing_extensions-4.11.0
                       commit: unknown

[jrkarnes]

I didn't have a field where it makes sense to provide additional context after the example_code portion, so I have opted to do so here.

In trying to resolve this, I did a little digging and have a few more things to report.

Dismantling the Forward Reference

I thought to try inverting the order of my classes to avoid a forward reference thinking that this would solve the issue. It did not. This code produces the same error as was submitted:

from pydantic import BaseModel
from typing import List, Optional

class StatelessRule(BaseModel):
    pass

class StatelessRulesAndCustomActions(BaseModel):
    StatelessRules: Optional[List[StatelessRule]] = None

class RulesSource(BaseModel):
    StatelessRulesAndCustomActions: Optional["StatelessRulesAndCustomActions"] = None

def test_the_bug():
  # Create instances of the classes to trigger type checking
  rules_source = RulesSource(StatelessRulesAndCustomActions=StatelessRulesAndCustomActions(StatelessRules=[StatelessRule()]))

Check if Name Collisions Are Allowed
So then I got to thinking that maybe this is just a constraint about Pydantic that I was unaware of and wrote another test using a different, but related, section of my real code. The name collision is definitely allowed when not used with Optional[] as the below code will execute without any problems in pytest:

from pydantic import BaseModel
from typing import Dict, List

class CustomActionDefinition(BaseModel):
    PublishMetricAction: Dict[str, List[Dict[str, str]]]

class CustomAction(BaseModel):
    ActionName: str
    CustomActionDefinition: CustomActionDefinition

def test_property_name_collision_allowed():
    custom_action = CustomAction(
        ActionName="foo",
        CustomActionDefinition={"PublishMetricAction": {"bar": [{"baz": "buzz"}]}}
    )

Check if the problem is really a name collision
At this point, I went back to the original code and renamed StatelessRulesAndCustomActions to StatelessRulesAndCustomActionsModel and magically the error went away. The below code runs as expected.

from pydantic import BaseModel
from typing import Dict, List, Optional

class StatelessRule(BaseModel):
    pass

class StatelessRulesAndCustomActionsModel(BaseModel):
    StatelessRules: Optional[List[StatelessRule]] = None

class RulesSource(BaseModel):
    StatelessRulesAndCustomActions: Optional["StatelessRulesAndCustomActionsModel"] = None

def test_the_bug():
  # Create instances of the classes to trigger type checking
  rules_source = RulesSource(StatelessRulesAndCustomActions=StatelessRulesAndCustomActionsModel(StatelessRules=[StatelessRule()]))

[Viicos]

This seems to be the same issue as: #7327, #8240, #7309 (see #6646 (comment) for an explanation).

See also #9093 (comment), could be the same issue you are facing

[sydney-runkle]

Closing as a duplicate, based on the issues that @Viicos mentioned above. Thanks!

[jrkarnes]

@Viicos

I read through the source material you provided and I do agree that this is a duplicate; however, I'm having a hard time reconciling why having the Optional[] type surround the value of the property sharing the same name is what causes this in the current version of Pydantic. If the behavior of the Optional[] type is responsible for this presentation of the collision error, I would wonder if the root cause of Python's typing hints is actually the root cause.

Do you have any insight into why the name collision was perfectly fine without Optional[] as I showed in my addenda?

[Viicos]

Not sure, might need to double check, but I think this comes from what I described here: #8243 (comment).

I'll probably need to take a better look at your code example, but could you confirm this matches your use case?

[jrkarnes]

Since Optional[...] is shorthand for Union[..., None] I think that it does.

On this same train, now that Union types are a thing in Py3.12, the syntax of date: date | None = None from some of your other examples would probably also run into this issue