Replies: 2 comments
-
This makes sense. The Moderate Amount of Vitessce Integration option seems great as a first step. Once that is in place, a follow-up pull request could add support for performing the cell set creation based on the selected regions. Since I can envision there being both file-based and user-created regions, I would store user-created regions using a new coordination type Then you could obtain the user-created regions from the useCoordination hook in |
Beta Was this translation helpful? Give feedback.
-
Cool! Glad that you're into the idea. Thanks for the pointers on where to start. I'll post back if I have any questions once I get started. |
Beta Was this translation helpful? Give feedback.
-
Proposal
I would like to propose adding functionally for selecting and persisting regions of interest on an image to Vitesse. Currently you can select a region to generate a cell set on an image, but the selected does not persist on the image. Regions can be loaded via bitmask images. However, cell sets cannot be created from regions loaded from bitmask images and users cannot dynamically add new regions. See attached video for an example of the proposed functionality in Mantis Viewer.
We have some datasets where pathologists have manually annotated tumor regions on an image, and we also have some data pipelines that try to automatically identify interesting or useful regions. It would be nice to load the regions into Vitessce and have cell sets automatically generated instead of having to pass it through another script to have the cell sets formatted for Vitessce.
Being able to persist and export regions on an image can also be useful for identifying regions that either should or should not be included in downstream analysis. Once those regions have been selected on an image they can then be exported as a labeled/bitmask tiff, which then can be used in downstream scripts or pipelines.
I can imagine that you might not want to add and support all of this functionality to Vitessce. I can imagine a few different possible solutions where some new functionality is added to Vitessce to support this use case and the remainder of this functionality is handled by an outside application
Least Amount of Vitessce Integration
For the solution that makes the least amount of changes to Vitessce, Vitessce would add the ability to display custom deck.gl layers and would either add a callback (maybe a generic onEvent or onUserInteraction) that would return the pixels or outline of a region selected or would add that information to the onConfigChange callback somehow.
In this case the user would select a region on the image, the outside application would get the coordinates, generate a new cell set, and then add the region to a custom deck.gl layer.
Moderate Amount of Vitessce Integration
For the second solution, Vitessce would add a new mode where regions could be selected and persisted on the image. Once selected, the outside application would get the coordinates of the region via onConfigChange and generate a new cell set. The outside application would manage keeping cell set and region color and visibility in sync. On reload the outside application would provide the coordinates for the regions and Vitessce would render those regions.
Most Vitessce Integration
For the final solution, Vitessce would support all of the described functionality, and changes could be read via onConfigChange to store in the outside application and loaded via config to load any stored regions into the application. In this solution, Vitessce would allow users to create and persist regions on the image, generate cell sets from persisted regions, and link color and visibility between regions and cell sets generate from those regions.
Conclusion
Hopefully this makes sense! Let me know if there are any questions. Also, let me know if you approve of any of the above solutions or would approve of any with modifications. If you do I will create new proposed features under issues for them and work on adding them to Vitessce.
Jun-16-2022.14-08-28.mp4
Beta Was this translation helpful? Give feedback.
All reactions