Custom and transferable stain vectors (brightfield)

[finglis]

Feature request

Is your feature request related to a problem? Please describe.
Currently setting stain vectors or adjusting pre-set ones (like H&E) requires scripting and being run for each image open (or project) to ensure they are all the same. This can get complicated particularly when dealing with multiple different brightfield (other) stain types in one project.
Additionally the ability to share these stain vectors would be useful, other than sharing the script.

Describe the solution you'd like
This request is for a way to create these custom stain vectors and retain them within QuPath either as a setting or within the project.
Additionally, being able to share them with others and allow for them to be imported into QuPath via drag and drop would be very beneficial.

Describe alternatives you've considered
Scripting is currently the best option for doing this, but this makes it less accessible to users that are not familiar with scripting. Additionally, the scripting method would require additional processing steps that could be reduced.

Additional context
May of missed key design points from discussion, feel free to edit/add!
Spoke about the same sharing functionality being used for fluorescent images but would only really be useful for those with many channels (which also currently uses scripting to set).

[petebankhead]

Thanks @finglis the more I think about this, the more I think we really need this.

Relatedly, I wonder if we could build on the idea to make auto-estimating the image type smarter?

My first (and entirely untested) idea would be to

• loop through all the known stain vector combinations
  • apply the stain vectors to an image
  • and calculate the mean absolute value (for example) of the residual
  • select the vectors where the error is minimized

This would only be suitable for 2 stains and might not handle artifacts very well, so I imagine it would need some refinement. But perhaps we can incorporate more prior knowledge (e.g. that stains should be negative) while still keeping the method simple and fast to compute.

The goal would be to enable a user to have the (probably small) number of stain vectors that are relevant for their images based upon their image sources/scanners, and have QuPath make more intelligent suggestions which to use - rather than relying upon scripting or using Estimate stain vectors on every image.

But I guess any auto-estimate is likely to be wrong sometimes. So some other related ideas:

• We could incorporate stain selection into the "Project import" dialog (alongside where we set the image type) to give another way of ensuring an image batch has consistent stain vectors set.
• We could add a 'Run for project'-like command that specifically exists to transfer the image type and stain vectors from the current image to all images in the project. This can already be scripted easily, but isn't very approachable for people who really don't want to handle scripts.

Logging the ideas here for consideration when we come to tackle the original enhancement, and in case any are worth discussing.

[MichaelSNelson]

Sounds like a great option, as long as it is not the default. Likely to be a somewhat slow process for a lot of images.

It would also be nice to have the stain vectors be (potentially) separated from some of the analysis options like the pixel classifier. Large project often have batch effects of samples from different scanners that require different stain vectors. Right now, I believe that the stain vectors are included in the pixel classifier, so no matter what adjustments are made to any given image, even though the deconvolved pixel values might be the only inputs fed into the pixel classifier, the classifier will use one set of stain vectors when performing the classification.

Is that still the case? If different training images are used via "Load training", each of which has different color vectors (for the same stain type!), I am guessing it currently ignores those other stain vectors in favor of the ones in effect in the image where the training process started.

[petebankhead]

Hi @MichaelSNelson yes, as far as I recall the stain vectors are still the ones used in training. Nothing was changed regarding that recently, and nothing can easily be changed because of compatibility considerations.

I thought about it at lot at the time, and I think both approaches are justifiable. If the stain vectors weren't stored with the pixel classifier, then the classification could be nonsense in some cases (e.g. if an image is misidentified as H&E, but the user doesn't notice). Although maybe that specific example would fail with an exception instead - I'm not sure.

Still, I think it would be better if we could support both fixed stain vectors and using the current ones with the pixel classifier - so we should look into that as well.