let people know about python lls deskew deconv code

[VolkerH]

Hi,
just wanted to make you aware of this repo. Still very early stages but enough there to bring it out in the open. Still uploading files in the coming days/weeks (travelling at the moment)q
@jni @tlambert03 @maweigert @uschmidt83 @eric-czech

[maweigert]

Looks great! Didn't know about flowdec - definitely need to check it out :)

[VolkerH]

I just updated the notebooks, btw (while you were writing your comment)

[VolkerH]

@maweigert
And I should mention that the flowdec people just discovered gputools
hammerlab/flowdec#12 (comment)

[eric-czech]

Very cool @VolkerH !

Iterating with smoothing until a single peak local max is left is a nifty trick, haven't seen that before. Do you think there are advantages to that over setting num_peaks in peak_local_max? I was just thinking about how you might extend that to multiple beads in the same image and I'm wondering what the pros/cons are to smoothing + peak local max finding vs smoothing + thresholding + finding intensity weighted centroids (which might give adequate subpixel translations for bead centers?) a la this script.

And itkwidgets looks awesome -- definitely going to give that a try.

Some last thoughts on flowdec related stuff that may be helpful:

• The PSFs can be smaller than the images but will get zero-padded up to the image size around the center (though the voxel dimensions are assumed to be the same and it doesn't do subpixel centering if an odd number of voxels need to be added along any one axis)
• Those times for 10 iterations on images of that size seem a bit on the slow side so you might be able to get better performance from RichardsonLucyDeconvolver(n_dims=3, start_mode='input', pad_mode='none'). It won't make a difference if the volume already has dimension lengths that are powers of two, but it might if it's not exactly 512x512x64 (which is about what it looks like in the notebook?)

Thanks for sharing that!

[VolkerH]

Hi @eric-czech,

thanks again for these comments. To be honest, i didn't even look at the options of peak_local_max .
The iterative smoothing was just based on the assumptions that

1. there is a single bead in the volume (I know that David captures them like that),
2. the maximum that remains longest corresponds to the centre of the PSF,

and using the property of the Gaussian that it does not shift the location of existing maxima or create new ones.

The iterative smoothing is probably unnecessary costly but I'm only doing this once to calculate the PSF.
Sub-pixel accuracy can probably be attained with this method by simply upsampling the image first. A better way would be to fit a distribution directly to the gray values and obtain an estimate for the centre and sigma of the fitted Gaussian.

Thanks for the link to the ExtractPSF fiji script. I see they are doing the reverse deconvolution (Distilling) with the bead size. Something I should probably add. I just saw @imagejan last week, I wish I had seen this earlier than I could have gotten some suggestions from him in person. Tagging him, maybe he has some comments.

Much of this code should already exist for single molecule localization, which also has things like multi-emitter fitting etc.

I will try your suggestions about speeding up things.

[eric-czech]

Makes sense. @bnorthan might be interested in what you guys are trying to do to since I think he wrote the original script and might be familiar with other real-time deconvolution efforts in the open source world.

FWIW, I'm getting about 370ms to run 10 iterations on a 512x512x64 image/psf (on a GTX 1080). How low would you have to get that for your use case? I think the biggest room for improvement in speed would be in implementing a non-circulant form of RL that would also allow the PSF to be smaller than the image, which is something I've been thinking about working in anyhow but have been too lazy to do so as of yet (it also helps alleviate some border effects). Anyways I'd be happy to help you try to hit a performance goal if you have a sense of whether or not that's close, or to at least see how far a TensorFlow-based implementation can be pushed (I'm sure there are some other minor inefficiencies in there too since real-time wasn't my original objective).

For reference:

[VolkerH]

Cool, thanks for these benchmarks. That's much faster than I anticipated.
At the moment, faster offline processing was the goal (our users are typically processing overnight).
I always thought that deskewed near real-time visualization should be possible if we can get the data from the camera caputre (happens in Labview) to a a viewer such as spimagine or Big-Data-Viewer.
I didn't think that deconvolution would be possible in that time frame but I'm happy to be see these benchmarks on the 1080.
See the discussion here: #11
For the coming days, I'm not sure whether I'll have much time to work on this, but I hope to get started again in a couple of weeks.

[imagejan]

@VolkerH thanks for sharing this repo!

Regarding the ExtractPSF.py script in imagej-scripting, @bnorthan is the one to be tagged (my contributions to the script were only cosmetic changes...).

[eric-czech]

No problem @VolkerH , that's much closer to our use case too.

[VolkerH]

Closing. Discussions can be continued in other issues.