Image loading and requirement for transpose()

[nicoloesch]

Hi everyone,

I have two questions regarding ScalarImage/Image as I am currently looking into a better way to load my data. I am using mostly BraTS but I also test my model with other datasets, that are mainly 2D-images.

1. My first question (less important as it has an easy workaround) would by, why torchio.io.ensure4d is not called if a tensor is specified whilst instantiating the Image (and child classes of it) but it is called when I specify the path instead? I am aware that torchio is not designed to support 2D images but rather volumetric data but would be a nice-to-have.
2. The other and more important question revolves around how the images are loaded. The reason why I am questioning this is that I also plot some of my results and they are all 90° degrees rotated (but also flipped due to the transpose operation - see following text). After investigation I found the reason why but did not have an explanation for it myself: torchio.io.sitk_to_nib in the first line specifiying sitk.GetArrayFromImage(image).transpose() (presumed the standard file reader based on SimpleITK is used). My question now would by why one needs to call .transpose? This results in the case of BraTS from 4x155x240x240 to 240x240x155, however with the caveat that both width and height are switched (but not obvious as both are 240). Wouldn't it suffice to just push the height dimension (155) to the end over .transpose(0,2,3,1)? Am I missing something here?

I guess some of it could be based on the Nifty format and how each axis contributes to one spatial dimension, with the BraTS images having a bad format?

Thanks for any help,
Nico

---

python: 3.9
torchio: 0.18.90 (not 91 but I assumed this would still be included)

[romainVala]

Hi
about your second point, I am not sure to fully understand the problem.
If this is a plotting issue, it then depend on how you choose to plot the data ... then you must give a more explicit example so that we can reproduce
This transpose() is quite handy to understand. It is related to the internal convention of data voxel order which is different between sitk and nibable. (the one you mention appear in io.py in function sitk_to_nib right ?)
My understanding is that torchio choose the nibable convention for the "interne" data representation (ie == the voxel order in the data key of the torchio Image) this is why there is a transpose here.
Now if you want to plot the data you need to be aware of this "internal" choice. My guess is that if you see a transpose in your plot (90°rotation) then you need to transpose the data before plotting them ...
does it make sense ?

[nicoloesch]

Hi,

Thanks for the answer! Seems like I was correct in the assumption of that it is an nibabel/sitk thing in terms of the transpose. I am currently also using the transpose again before plotting to get the images back to the 'correct' orientation, however I was just curious as to why this is done in general.

Cheers,
Nico

[romainVala]

We need to make a choice for the internal voxel order (nibabel or stik) . The specific choice either on or the other is arbitrary. But once you made a choice you need to stay coherent and add this transpose when needed ...

[fepegar]

My first question (less important as it has an easy workaround) would by, why torchio.io.ensure4d is not called if a tensor is specified whilst instantiating the Image (and child classes of it) but it is called when I specify the path instead?

I think the idea is that ensure4d tries its best to make to load images and convert the shape into the appropriate one, whereas when you pass a tensor directly as an arg, you have full control of what's being passed and it's your responsibility to use the appropriate shape. Does that make sense?

Regarding the array dims order, see "The order of index and dimensions need careful attention during conversion" in https://simpleitk.org/SimpleITK-Notebooks/01_Image_Basics.html