Add support for specifying axes for the navigator and "streak" navigator

[ericpre]

@jlaehne, as mentioned in LumiSpy/lumispy#204 (comment), this PR allow plotting streak images with additional navigation dimension.

Progress of the PR

• Simplify setting navigator,
• make navigation dimensions consistent between lazy and non-lazy signal,
• add support for "streak image" navigator,
• add support for specifying axes of the navigator,
• update docstring (if appropriate),
• update user guide (if appropriate),
• add an changelog entry in the upcoming_changes folder (see upcoming_changes/README.rst),
• Check formatting changelog entry in the readthedocs doc build of this PR (link in github checks)
• add tests,
• ready for review.

Minimal example of the bug fix or the new feature

import hyperspy.api as hs
import numpy as np

shape = (3, 4, 5, 10)
data = np.arange(np.prod(shape)).reshape(shape) + np.random.default_rng().random(shape) * 100
s = hs.signals.Signal1D(data)

s.plot(navigator="streak")

s.plot(navigator_axes=[0, 2])

[ericpre]

@CSSFrancis, this PR changes which navigation axes are displayed in the image navigator, because it simplify the code - at the least with the approach currently used! ;)
Given a signal <a, b, c | d >, it displays the b, c axis instead of a, b. I am not convinced this is a good changes, what do you think?
For example, when stacking signals with navigation of 2, you get the following: <a, b, stacking_axis | d >, in which case the changes of this PR is not favorable.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.58%. Comparing base (950741a) to head (8d9e544).
Report is 49 commits behind head on RELEASE_next_minor.

Additional details and impacted files

@@                  Coverage Diff                   @@
##           RELEASE_next_minor    #3309      +/-   ##
======================================================
+ Coverage               80.31%   80.58%   +0.26%     
======================================================
  Files                     147      147              
  Lines                   21854    21871      +17     
  Branches                 5140     5146       +6     
======================================================
+ Hits                    17553    17624      +71     
+ Misses                   3088     3033      -55     
- Partials                 1213     1214       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[CSSFrancis]

@CSSFrancis, this PR changes which navigation axes are displayed in the image navigator, because it simplify the code - at the least with the approach currently used! ;) Given a signal <a, b, c | d >, it displays the b, c axis instead of a, b. I am not convinced this is a good changes, what do you think? For example, when stacking signals with navigation of 2, you get the following: <a, b, stacking_axis | d >, in which case the changes of this PR is not favorable.

Hmmm, in my case I have a bunch of 5D STEM data where the axes are <x, y, time | kx, ky>. We could represent the data as
<time, x, y| kx, ky> as well but because of the large size of the data it is difficult to rechunk the data into that form! Because we have a huge binary file that is read out it takes a long time to read the data out in a non sequential way. In our case we really want three things plotted [1d plot of total intensity vs time, 2d plot of real space xy, 2d plot of reciprocal space xy]

My original thought was that we should make it so that if someone passes a navigator which has a signal and navigation axes then that plot should become the interactive navigator with the signal updating and the navigator static. This gives the most flexibility and you could possibly stack navigators to have 6,7 etc dimensions.

[CSSFrancis]

I guess that doesn't really get at the core of the problem which is that sometimes you want to have a plot that updates along the outer most dimension. You could add in a transpose_navigation function which switches to a more numpy like representation. This is kind of inline with how you can change the origin in matplotlib to align with numpy.

[ericpre]

I guess that doesn't really get at the core of the problem which is that sometimes you want to have a plot that updates along the outer most dimension.

Yes, this is the right way to approach it and it makes the implementation more straightforward. This is done, as per example above.

My original thought was that we should make it so that if someone passes a navigator which has a signal and navigation axes then that plot should become the interactive navigator with the signal updating and the navigator static. This gives the most flexibility and you could possibly stack navigators to have 6,7 etc dimensions.

Actually, it may be possible without too much trouble, but we can leave that for another day! 😉

[ericpre]

@jlaehne, do you think that this PR would help with the representation of streak data as discussed in LumiSpy/lumispy#20?

@LMSC-NTappy, does it look useful to you?

[Attolight-NTappy]

@ericpre Definitely looks useful to me. Thanks!

Just a question: would s.plot(navigator_axes=[0,2]) produce streak-like navigation with axes 0 considered as the spectral axis or would it raise an error ?

Also, that's just a detail but I am not certain that naming the "streak" denomination is optimal as this way of displaying the data will have uses beyond streak maps.

[ericpre]

Thanks @Attolight-NTappy!

@ericpre Definitely looks useful to me. Thanks!

Just a question: would s.plot(navigator_axes=[0,2]) produce streak-like navigation with axes 0 considered as the spectral axis or would it raise an error ?

In the example above, axis 0 is a navigation axes and axis 2 is the first signal axes, using "name" of the axis is more clear - I updated the example accordingly. Using an axes from signal space is only supported for Signal1D - because for Signal2D, it is not sensible as it would be the same as the signal plot!
This made me think that we could also plot the reversed order of navigation axes. I fixed it and updated the example to illustrate it! 😃

Also, that's just a detail but I am not certain that naming the "streak" denomination is optimal as this way of displaying the data will have uses beyond streak maps.

Indeed, I was not sure if "steak" was a good wording, but I couldn't think of anything better. In the context of EELS, I have seen it being used to visualise energy shift over time.
Another application of this visualisation is momentum resolved EELS, for example: https://www.nature.com/articles/s41586-019-1477-8/figures/1 (for one direction only).
@francisco-dlp, any idea what would be a better wording?

[jlaehne]

@jlaehne, do you think that this PR would help with the representation of streak data as discussed in LumiSpy/lumispy#20?

Sorry I was away and am rather busy right now. But I will do my best to have a look and play with some real data with this branch asap.

[Attolight-NTappy]

In the example above, axis 0 is a navigation axes and axis 2 is the first signal axes, using "name" of the axis is more clear - I updated the example accordingly. Using an axes from signal space is only supported for Signal1D - because for Signal2D, it is not sensible as it would be the same as the signal plot!
This made me think that we could also plot the reversed order of navigation axes. I fixed it and updated the example to illustrate it! 😃

Makes sense to me !

Unfortunately, I don't have a suggestion for another word than "streak" on top of my head. Something like "linescan" maybe? I think it would be useful to use a word that links intuitively with "map"

[CSSFrancis]

@ericpre This looks really good!

As far as implementations go I think that this is much cleaner than previous which is good, but we might want to think about how we want to do this in the future to be a bit kinder to ourselves.

I think that ultimately we want a couple of things:

1. Markers which update with the navigator.
2. Multiple Navigators for higher dimensional datasets.

I think that lends itself to passing navigators with both signal and navigation axes so it might be good to leave that as an option.

[CSSFrancis]

Is there a reason to convert the nan to a number for plotting? Nan's are actually quite useful when you are overlaying two plots as the nan part of the image is transparent.

Of course we don't currently have a way to layer images but I actually have thought about something like passing two navigators and overlaying the two images. For something like orientation mapping it might be nice to overlay the phase with an alpha of like .5 and then have the values where no phase is found be nan.

[ericpre]

The reason is that if there is a single np.nan, the sum will np.nan:

import numpy as np
a = np.empty(10)
a[0] = np.nan 
print(np.sum(a))

gives

nan

By default, np.nan_to_num set the nan values to zero but it is possible to set to something else, which should give enough flexibility for what you have in mind, when implemented.

[CSSFrancis]

Is it going to be weird if we deprecate this and then bring this back at some point?

I would like to support navigators with signal dimensions eventually.

[ericpre]

I can have a look: It may possible without too much trouble. Can you elaborate how do you see working?
Something as:

s = hs.signals.Signal1D(np.arange(5*10*20*40).reshape((20, 10, 5, 40)))
s2 = s.sum(-1).as_signal1D(-1)
s.plot(navigator=s2)

Where the s pointer will also be added to the signal dimension of s2?

As we need to check the consistent in shape between the signal dimension of the provided navigator signal (s2) and the corresponding navigation dimension(s) of s, how do we define which navigation dimension of s should it be matched to?

[CSSFrancis]

Okay so I'll start with the example that is most relevant to me:

If you have a time resolved 4D STEM dataset you might want something like this:

import hyperspy.api as hs
import numpy as np
time_series_navigator = hs.signals.Signal2D(np.arange(5*10*10).reshape((5, 10,10))) # <Signal2D, title: , dimensions: (5|10, 10)>

full5Ddata = hs.signals.Signal2D(np.arange(5*10*10*128*128).reshape((5, 10,10, 128,128))) # <Signal2D, title: , dimensions: (10, 10, 5|128, 128)>
updating_nav_pos = np.empty(5, dtype=object)
for i in np.ndindex(updating_nav_pos.shape):
    updating_nav_pos[i] = np.random.random((np.random.randint(3,5), 2))
p1 = hs.plot.markers.Points(updating_nav_pos)

full5Ddata.plot(navigator=time_series_navigator) # plots three figures the navigator for time_series_navigator, the signal for time_series_navigator and the signal for full5Ddata 
time_series_navigator.add_marker(p1) # add an updating navigator to the navigator plot

With the output being something like:

Navigator 1 &2:

Signal:

[CSSFrancis]

This shouldn't be too difficult from the standpoint of writing new code, but tracking everything is going be a bit of a pain so I might end up coming back to it in a bit once I get the time.

[ericpre]

@CSSFrancis, I removed the deprecation.

[jlaehne]

Finally getting to this. Looks like it indeed adds flexibility to the navigator plotting, but so far I could not reproduce scans of streak images with that to the degree I want. For example creating a Signal2D map from a single streak image s (and modulating intensities a bit to vary the color in the navigator)

>>> map = hs.stack([hs.stack([s]*3)]*3)
>>> map
<Signal2D, title: Stack of Stack of photon_counting, dimensions: (3, 3|672, 512)>

I get a navigator / signal image:

I could not reproduced something similar, when having one of the signal axis transformed to a third navigation axis.

Of course it could be interesting to have the streak image be another navigator (but then one of the axes is a signal axis) from which to extract spectra or transients with an extra slider. However, that is something easily obtained for a single streak image without a nexted navigator.

[jlaehne]

A bug I had when plotting with this PR: In the case of the navigator map and image shown above, when I move the position on the navigator, the axes labels in the image plot are suddenly lost.

[ericpre]

This shouldn't be too difficult from the standpoint of writing new code, but tracking everything is going be a bit of a pain so I might end up coming back to it in a bit once I get the time.

@CSSFrancis, I had a look and it is getting a bit messy with the current implementation. I think that this should be done after the plotting have refactor to allow resizable pointer and to decoupled from the AxesManager.indices - there was some discussion on this previously (online and offline).

A bug I had when plotting with this PR: In the case of the navigator map and image shown above, when I move the position on the navigator, the axes labels in the image plot are suddenly lost.

@jlaehne: unrelated to this PR, recently, I noticed that the colorbar intermittently disappear and this seems to be related to blitting and the issue you are referring may falling in this case. Can you please provide the code to reproduce it?

[CSSFrancis]

@CSSFrancis, I had a look and it is getting a bit messy with the current implementation. I think that this should be done after the plotting have refactor to allow resizable pointer and to decoupled from the AxesManager.indices - there was some discussion on this previously (online and offline).

@ericpre I would agree. I think the plotting could be cleaned up a bit. I looked at it too and realized that it is a bit more complicated than I thought as well to add that feature.

Any chance you want make an issue and maybe we can start to develop a step by step road map for things we want to do with the plotting?

[ericpre]

Any chance you want make an issue and maybe we can start to develop a step by step road map for things we want to do with the plotting?

#3325