Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential Memory leak when using model #6109

Open
noki00003 opened this issue May 7, 2024 · 0 comments
Open

Potential Memory leak when using model #6109

noki00003 opened this issue May 7, 2024 · 0 comments
Labels
bug Issue describes a potential bug in ml-agents.

Comments

@noki00003
Copy link

noki00003 commented May 7, 2024

I am experiencing a bug, that when I have finished training my model and I want to use it then, I put the model into the Model Field in Unity. I do this for 16 Agents in a single environment. The first run there are no problems. Then I stop the run and change all models except for one to "Inference only" (the first run was "Default") and the one remaining agent to "Heuristic only", as I want to play myself against the other agents.
Then I get the warning:

"Found unreferenced, but undisposed BurstTensorData that potentially participates in an unfinished job and might lead to hazardous memory overwrites
UnityEngine.Debug:LogWarning (object)
Unity.Sentis.D:LogWarning (object) (at ./Library/PackageCache/com.unity.sentis@1.2.0-exp.2/Runtime/Core/Internals/Debug.cs:72)
Unity.Sentis.BurstTensorData:Finalize () (at ./Library/PackageCache/com.unity.sentis@1.2.0-exp.2/Runtime/Core/Backends/CPU/BurstTensorData.cs:145)".

When I then change all the agents back to "Default" as in the first run, the warning still occurs. I have looked for solutions for a long time on the Internet, but there is only one solution, where I do not understand where in the code I need to change something
( AlexRibard/Barracuda-U-2-NetTest#3 ).

To reproduce this error you need to follow these simple steps:

  1. open up the default PushBlock environment provided by mlagents
  2. hit run once and then everything should run as desired with the provided model asset
  3. stop running and change only the agent of the first environment to "heuristic only"
  4. hit run again and then there should be the warning in the console

Edit: You can simply hit run then stop and then run again, I just found out
Edit2: It does only happen in release 21 (3.0.0) and not in release 20 (2.3.0)

I am not really sure, whether I tested this excact scenario with Torch-CPU before, but right now I am using Torch-CUDA 12.1 with torch version 2.3.0.
My other specs:

  • Windows 11
  • Unity 2022.3.26f*
  • ml-agents 1.0.0
  • in unity mlagents 3.0.0 (stable release 21)
  • Python 3.10.12

memoryLeakIssueUnity

@noki00003 noki00003 added the bug Issue describes a potential bug in ml-agents. label May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue describes a potential bug in ml-agents.
Projects
None yet
Development

No branches or pull requests

1 participant