-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory leak in pydantic model_validate from sqlalchemy model #9429
Comments
Thanks for the report. I haven't taken an in-depth look yet, but maybe @davidhewitt has some insight. |
Hi, I'll wait for his feedback. |
Hello @davidhewitt do you have any hint to share ? |
I have taken a brief look but I cannot reproduce this on any combination of sqlalchemy 1.4 or 2, Python 3.11 or 3.12 or pydantic 2.7 or main. If you have another example of what you think is the problematic interaction I can take a further look. |
Hello, Thank you for testing it. You mean that the test passes on your configuration ? We have sql alchemy 1.4, pydantic 2.7.1 and python 3.11.5 on windows. (servers on red hat) My expected result would be that all memory is freed up after the pydantic object if removed. If you could share your python version, it could be nice. Would help me to test if something changes depends on that. Rergards, |
To check, you're testing on Windows? I tried 3.11 and 3.12, both on Ubuntu. |
(Yes, the test passes for me) |
which version of 3.11 are you using please ? |
3.11.6 |
Hi, We tested on windows with 3.11.5, 3.11.6 and 3.11.9 and it's failing each time. On our linux server, it passes on 3.11.5. |
I will try to reproduce on Windows some time soon, most likely next week. |
Any luck here for anyone? I've also tracked a memory leak down to model_validate and am only using pydantic with json objects. Tracked using memory_profiler
Weirdly enough if I uncomment the commented lines the leak goes away, even though |
Hi, We still have it, and so far, we were unable to find how to resolve it. |
Initial Checks
Description
When creating a pydantic object from an sqlalchemy object using model_validate, I see only increase of memory usage.
I deep dived the issue and I think there might be a problem with model_validate method.
The following test create pydantic and sqlalchemy model the way fastapi does.
When used that way, the memory is never released.
When using model_validate(orm_object.dict) the memory is released, but I don't think we are supposed to be forced to create object from dict, and it doesn't work weel with nested objects.
Do you have any idea ?
regards,
Pierre
PS : on my application, it results in 4-5GB of ram being used during DB saving... never released.
Example Code
Python, Pydantic & OS Version
The text was updated successfully, but these errors were encountered: