Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rendering volumes of some resolutions crashes when using itkwidgets.view after using shapeworks in a notebook #1168

Closed
akenmorris opened this issue Mar 23, 2021 · 23 comments

Comments

@akenmorris
Copy link
Contributor

akenmorris commented Mar 23, 2021

This used to be "getting-started-with-grooming-segmentations notebook crashes on MacOS," but lots of investigation has uncovered the actual issue, possibly due to incompatible shared libraries.

Clean system, clean conda.

This is the cell that crashes:

shapeSeg = shapeSegList[10]
itkw.view( image          = sw2vtkImage(shapeSeg),
           slicing_planes = True,
           axes           = True,
           rotate         = True,
           interpolation  = True)
Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x000000014b76fff3
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [0]

VM Regions Near 0x14b76fff3:
    MALLOC_LARGE           000000014b50a000-000000014b769000 [ 2428K] rw-/rwx SM=PRV  
--> 
    MALLOC_LARGE           000000014b770000-000000014d863000 [ 32.9M] rw-/rwx SM=PRV  

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_platform.dylib      	0x00007fff564c85e6 _platform_memmove$VARIANT$Nehalem + 486
1   libvnl.dylib                  	0x000000010ae40e6b vnl_vector<double>::operator=(vnl_vector<double> const&) + 139
2   _ITKCommonPython.so           	0x000000014ec0d18b vnl_vector<double>::operator=(vnl_vector<double>&&) + 65
3   _ITKCommonPython.so           	0x000000014eadec9d 0x14de03000 + 13483165
4   _ITKCommonPython.so           	0x000000014e869165 0x14de03000 + 10903909
5   _ITKCommonPython.so           	0x000000014ec611a7 itk::ProcessObject::UpdateOutputInformation() + 339
6   _ITKCommonPython.so           	0x000000014e47d8fe 0x14de03000 + 6793470
7   _ITKImageGridPython.so        	0x00000001538b5619 itk::ProcessObject::UpdateOutputInformation() + 117
8   _ITKImageGridPython.so        	0x00000001538b5ebc itk::ProcessObject::UpdateLargestPossibleRegion() + 18
9   _ITKCommonPython.so           	0x000000014e10726b 0x14de03000 + 3162731
10  python                        	0x00000001044c8843 _PyMethodDef_RawFastCallKeywords + 131
11  python                        	0x00000001044c81d6 _PyObject_FastCallKeywords + 598
12  python                        	0x0000000104600bb7 call_function + 455
...
@cchriste
Copy link
Contributor

Yup, I'm trying to run it now and hitting the same crash, right here:

itkw.view( image          = sw2vtkImage(shapeSeg),
           slicing_planes = True,
           axes           = True,
           rotate         = True,
           interpolation  = True)

@cchriste cchriste self-assigned this Mar 24, 2021
@archanasri
Copy link
Contributor

archanasri commented Mar 24, 2021

Simplified the issue and it looks like Image::resample is the culprit here. After resampling, ellipsoid_05 loads the viewer while ellipsoid_00 causes the kernel to crash when trying to use itk.view()

Screen Shot 2021-03-24 at 11 02 20 PM

Screen Shot 2021-03-24 at 11 01 06 PM

@archanasri
Copy link
Contributor

notebooks.zip

@cchriste
Copy link
Contributor

After testing with this, something may be wrong with Image::resample wrt itkwidgets.view.
Images tried: ellipsoid_1mode/segmentations/ellipsoid_00.nrrd and ellipsoid_1mode/segmentations/ellipsoid_05.nrrd
-> 00 fails to be plotted when wrapped by pyvista in a vtk image after resampling
-> 05 works fine
=> both work fine when plotted by pyvista
I threw some debug spew into Image::resample and saw nothing out of the ordinary so far).

Here's a simple notebook if you want to experiment.
wishy_washy_resample.ipynb.zip

...and what it looks like empty when you load it:
Screen Shot 2021-03-24 at 7 59 23 PM

...and what it looks like when you run it. Change the 05 to 00 and run it again to crash when itkw tries to render.
Screen Shot 2021-03-24 at 7 55 29 PM

@cchriste
Copy link
Contributor

Transpose or... The Array That Could Be Rendered
If you recall, itkwidgets can render a numpy array directly. But it interprets that array as the transpose of what is passed, something it does not do when a vtk array is used. Therefore we typically transpose what we get from the image before wrapping it in a vtk array, and simply use the vtk array with itkwidgets to keep things simple.

I tried both this morning and unfortunately it doesn't make a difference. Transposed or not, numpy or vtk arrays, it all crashes when trying to rendering using itkwidgets after Image.resample.

@cchriste
Copy link
Contributor

Copy or... Am I Repeating Myself?
The array coming from the Image might be the problem. Maybe it's pointing to invalid memory. So let's create a copy!
After copying the array--in both order='C' and order='F', transposing, modifying, standing on my head while composing the functions, and wishfully thinking about it all, it still crashes every time.

This suggested the size of the array itself might be a problem. One way to confirm this was to call resample with the same spacing ([1,1,2]), which worked fine. Another way was to create a bogus numpy array of said questionable size. And...
Drum roll please...
presenting...
for the first time ever... (in GitHub)
A shapeworks-free repro!

array = np.ndarray([109, 77, 204])
itkw.view(image = array)

So go put that in your notebook and smoke* it.

*smoke implying crash, not advocating for actual inhalation of anything, note this is a non-smoking facility, not valid with any other offer, ...anyway, I hope someone is smiling, 'cuz this ain't our problem! Except that it is. 😞

@archanasri
Copy link
Contributor

array = np.ndarray([109, 77, 204])
itkw.view(image = array)
array = np.ndarray([109, 109, 204])
itkw.view(image = array)
array = np.ndarray([109, 109, 109])
itkw.view(image = array)

All these examples render the itk viewer when shapeworks isn't imported.
It crashes when shapeworks is imported.

@akenmorris
Copy link
Contributor Author

Hmm, do we have some conda itk/vtk vs shapeworks itk/vtk library interaction going on?

@archanasri
Copy link
Contributor

I guess itkwidgets is using another version of itk

@cchriste
Copy link
Contributor

Order of import is not important, but order of usage is. This means if you import everything, but use itkwidgets before shapeworks, it crashes when you use shapeworks, and vice versa.

@cchriste
Copy link
Contributor

Since the solution to #1179 was simply "don't use itk," I propose we resolve this issue the same. Unless there is a better option at this time, let's remove itkwidgets from our notebooks.

@akenmorris
Copy link
Contributor Author

That sounds like a good plan for now. Is it reasonable to remove itkwidgets from the notebooks? What are we using it for?

@archanasri
Copy link
Contributor

We are using it to view the segmentations. I think we can use pyvista instead

@sheryjoe
Copy link
Contributor

Note that pyvista (as far as I know) does not respect image meta-information (origin, voxel spacing, axes direction), yet itkwidget does.

@cchriste
Copy link
Contributor

Note that pyvista (as far as I know) does not respect image meta-information (origin, voxel spacing, axes direction), yet itkwidget does.

We'll dig into this, but there must be some respect for origin since multiple volumes can be simultaneously plotted with different locations, and this image shows that spacing is used for plotting. Our sw2vtkImage function(s) simply didn't keep it (I know I changed this in some, but maybe those are in the new pybind branch).

Screen Shot 2021-03-26 at 1 34 32 PM

@cchriste
Copy link
Contributor

cchriste commented Mar 26, 2021

I want to suggest we release and push this to the next release since that one has many other python and notebook improvements, and essentially we'll be forced to rewrite them. Maybe we just let them be broken. It's a known issue in a public repository and we have a workaround in case any users run into it.

(We will still need to modify the getting-started-with-segmentations notebook so it completes without crashing, which simply means picking a different arbitrary image to demonstrate rendering.)

@cchriste cchriste changed the title getting-started-with-grooming-segmentations notebook crashes on MacOS rendering volumes of some resolutions crashes when using itkwidgets.view after using shapeworks in a notebook Mar 26, 2021
@sheryjoe
Copy link
Contributor

I agree. Let's push this to 6.1 release and I will update the release notes and the graphical abstract to de-emphasize python stuff.

@archanasri
Copy link
Contributor

archanasri commented Mar 26, 2021

After using an arbitrary image that doesn't crash the viewer, it still crashes at 3 more places that use the mean shape image.
@cchriste

@cchriste
Copy link
Contributor

I propose we exclude this notebook from the release except in the documentation.

@akenmorris
Copy link
Contributor Author

Works for me. @sheryjoe ?

@sheryjoe
Copy link
Contributor

@cchriste
Copy link
Contributor

pyvista showing logical coordinates
versus itkwidgets showing physical coordinate

Given issue #900, it's hard to imagine itk having ever showed physical coordinates.

For this image (with z-spacing = 2):

{
	dims: [93, 87, 94],
	origin: [-24, -19, -21],
	size: [93, 87, 188],
	spacing: [1, 1, 2]
}

The itk viewer can only show the unscaled version since otherwise it crashes.
Screen Shot 2021-03-30 at 3 57 34 PM

Here's a single pyvista volume showing both the unscaled ellipsoid and the scaled version.
Screen Shot 2021-03-30 at 3 56 20 PM

I don't really like the way the pyvista viewer shows bounds compared to the itk viewer.
But they're correct, and they're showing physical coordinates.

@cchriste
Copy link
Contributor

Closing since issues will go away for us when #1186 is complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants