Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for displaying stereo images in multiview #1669

Open
danellecline opened this issue May 6, 2024 · 3 comments
Open

Add support for displaying stereo images in multiview #1669

danellecline opened this issue May 6, 2024 · 3 comments

Comments

@danellecline
Copy link

Viewing stereo images would be very helpful for comparing model output side-by-side for robotics research applications.

Do you have an estimate of the level of effort this would require? I'm willing to consider doing this work but must balance it with several other projects. A rough estimate in days/weeks is helpful.

@jrtcppv
Copy link
Member

jrtcppv commented May 8, 2024

Hello, it would be really great if you added this feature. I would estimate around 120-160 hours of work to implement, using the image annotation view and multi-stream video annotation view as a guide. There may be some edits required to tator-js, which contains some of the lower level functionality. Specifically you may need to modify the image canvas element, as some modifications were needed for the video canvas to support grid views etc. in the multi-stream video annotation view.

In addition to the front end, there may be some modifications to the REST service required to represent multi-view images in the media multi dtype. This will likely only include safeguards, such as preventing combination of video and image. We believe the existing schema should have everything you need so do not expect any database migrations (changes to models.py). There may be a need to modify the tator-py utility for creating multi-view images as it currently expects videos only. Finally we would want to implement unit tests for the multi-image view.

I think we can take care of most of the items laid out in the second paragraph above; none of those are required for an initial capability. You could start by just making a Media object manually with the Django shell, set its component IDs to two images, and then start building the multi image view around that. If you can get there, we can help with the rest (REST safeguards, tator-py utility, and unit tests). And feel free to ask any questions here, or if you'd like we can set up a shared Slack channel.

@marktaipan-cvisionai
Copy link
Member

@danellecline - the [dev/1669-multi-images](https://github.com/cvisionai/tator/tree/dev/1669-multi-images) branch has a skeleton of the proposed changes. It includes a lot of the REST service multi dtype changes (the inclusion of the multi-image dtype) as well as an annotation-multi-image.js web element that mimics the annotation-multi.js video version with all the video-specific stuff removed. I'm hoping the branch helps with jumping into this feature development. Let us know if there are any questions, and thanks for taking a look!

@danellecline
Copy link
Author

Terrific. I will inspect this more closely and setup a dev environment next week. Many thanks @marktaipan-cvisionai.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants