Matplotlib runs out of memory

[milind-prajapat]

Bug report

Bug summary

Code runs out of the memory, uses all of the available RAM, causes notebook crash on google colab.

Code for reproduction

import numpy as np
import matplotlib.pyplot as plt

x = np.arange(0, 280, 0.1)
y = np.sin(x)

for k in range(100):
  fig, ax = plt.subplots(6, 2, sharex = True)
  fig.set_size_inches(37.33, 21)

  for i in range(2):
    for j in range(6):
      ax[j][i].plot(x, y)
  
  fig.savefig(f'figure{k}.png', dpi = 300)
  plt.close(fig)

Actual outcome

Able to save around 38 figures out of 100, and session gets crashed, since RAM gets full.

Expected outcome

Should save all of the figures, with causing no extra RAM usage.

Matplotlib version

• Operating system: Linux
• Matplotlib version (import matplotlib; print(matplotlib.__version__)): 3.2.2 / 3.4.1 (tried on both)
• Matplotlib backend (print(matplotlib.get_backend())): module://ipykernel.pylab.backend_inline
• Python version: 3.7.10
• Jupyter version (if applicable): 5.2.2
• Other libraries: numpy: 1.19.5
• Platform: Google Colaboratory

[timhoffm]

Running this on master in ipython, this finishes correctly and I don't see an increase in mem usage over time.

Please try stripping down your example to the minimum that is needed to reproduce the crash.

[milind-prajapat]

@timhoffm This issue is particularly caused on google colab.
Code is reduced to minimum that is needed to reproduce the crash.

[timhoffm]

This is related to the inline backend. Can be reproduced locally in a jupyter notebook using

%matplotlib inline

import matplotlib.pyplot as plt

for k in range(100):
  fig, ax = plt.subplots(figsize=(37.33, 21))
  fig.savefig(f'figure{k}.png')
  plt.close(fig)

The memory leak can be avoided by explicitly switching to the agg backend:

%matplotlib inline

import matplotlib.pyplot as plt
import matplotlib
matplotlib.use('agg')

for k in range(100):
  fig, ax = plt.subplots(figsize=(37.33, 21))
  fig.savefig(f'figure{k}.png')
  plt.close(fig)

[jklymak]

I think that is ipython's backend? From our point of view its just an agg buffer....

[timhoffm]

Indeed the inline backend is under ipython's control. So the issue should be posted to https://github.com/ipython/ipython.

[jklymak]

There probably is some way to flush the python output every step, but then you will have hundreds of plots on your page. I'll close here because I don't think there is anything matplotlib can do here.

[jklymak]

cross ref: ipython/ipython#7270

[tacaswell]

ipython/ipython#7270 (comment) <- pretty sure this is the correct explanation.

[milind-prajapat]

@timhoffm @jklymak @tacaswell Thanks for looking into it and resolving the issue. Solution works perfectly.