np.save failure for large array

[tashrifbillah]

Description

Python 2 fails to np.save a large array in this line.

$ /data/pnl/soft/pnlpipe3/Slicer-4.10.2-linux-amd64/python-realWithExtensions.sh

>>> np.save('/PHShome/tb571/dqc_tmp.npy', [img[0],img[0][...,1], '3'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/pnl/soft/pnlpipe3/Slicer-4.10.2-linux-amd64/lib/Python/lib/python2.7/site-packages/numpy-1.13.1-py2.7-linux-x86_64.egg/numpy/lib/npyio.py", line 509, in save
    pickle_kwargs=pickle_kwargs)
  File "/data/pnl/soft/pnlpipe3/Slicer-4.10.2-linux-amd64/lib/Python/lib/python2.7/site-packages/numpy-1.13.1-py2.7-linux-x86_64.egg/numpy/lib/format.py", line 576, in write_array
    pickle.dump(array, fp, protocol=2, **pickle_kwargs)
SystemError: error return without exception set

Reference

numpy/numpy#2396 and https://bugs.python.org/issue11564

As mentioned in the first reference, it should be fixed in Python 3.

Additional info

>>> np.save('/PHShome/tb571/dqc_tmp.npy', [img[0][...,:100],img[0][...,1], '3'])
>>> np.save('/PHShome/tb571/dqc_tmp.npy', [img[0][...,:200],img[0][...,1], '3'])
>>> np.save('/PHShome/tb571/dqc_tmp.npy', [img[0][...,:300],img[0][...,1], '3'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/pnl/soft/pnlpipe3/Slicer-4.10.2-linux-amd64/lib/Python/lib/python2.7/site-packages/numpy-1.13.1-py2.7-linux-x86_64.egg/numpy/lib/npyio.py", line 509, in save
    pickle_kwargs=pickle_kwargs)
  File "/data/pnl/soft/pnlpipe3/Slicer-4.10.2-linux-amd64/lib/Python/lib/python2.7/site-packages/numpy-1.13.1-py2.7-linux-x86_64.egg/numpy/lib/format.py", line 576, in write_array
    pickle.dump(array, fp, protocol=2, **pickle_kwargs)
SystemError: error return without exception set

Checked and confirmed no problem in Python 3.

Solution

Wait until Slicer 4.11 stable is released and potentially modify SlicerDiffusionQC to work with Python 3 (should be compatible already).