Memory leak in io.load_electron_counts

[ercius]

I often will use the same notebook to look through many datasets without restarting the kernel.

I often run ee = stio.load_electron_counts(fullPath) and each time I run this the memory increases.

Using del ee does not release the memory. Running garbage collection import gc;gc.collect() also does not release the memory.

Is there a way to release the memory?

[cjh1]

@ercius I will take a look, del ee should do it.

[cjh1]

Is it just with the stio.load_electron_counts(fullPath) call that you see the memory increase, as in is there anything else going on in the notebook?

[cjh1]

OK, I can recreate this in a simple notebook, will work out what is going on.

[cjh1]

So I was able to create a test case without stempy that shows the same behavior. It seems the problem only occur with the vlen type. I have raised an issue with h5py, we will see if they can shed any light on this, its possible its a bug in h5py.

[ercius]

Thanks for looking into this. Let's see what h5py says.

…

On Fri, Nov 6, 2020, 10:11 AM Chris Harris ***@***.***> wrote: So I was able to create a test case without stempy that shows the same behavior. It seems the problem only occur with the vlen type. I have raised an issue with h5py, we will see if they can shed any light on this, its possible its a bug in h5py. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#185 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADXORFAFB24A3VCUC3H4K4LSOQ36TANCNFSM4TL36EJQ> .

[ercius]

this is tracked in h5py/h5py#1747