[FEATURE] Image Bounding Box Annotations for vision & multimodal workflows #4739
Labels
area: ui
Indicates that an issue or pull request is related to the User Interface (UI)
team: ml
Indicates that the issue or pull request is owned by the Machine Learning (ML) team
type: enhancement
Indicates new feature requests
Hi team!
Given the awesome SpanQuestion new feature recently released, I'm tempted to ask if it's possible to have the same done for annotating regions of interests for images. It would be marvelous if there's a way to draw and label rectangle bounding boxes on an
ImageField
, similar to LabelStudio's bounding boxes. I love Argilla and hope this feature can be on the roadmap!Is your feature request related to a problem? Please describe.
As LLMs are increasingly becoming multimodal and many data workflows involve a mixture of text, document, and image data types, a very common task is to highlight specific regions in an image for annotation or other downstream processing. For a specific use case, suppose an LLM is tasked with detecting objects and return a bounding box of the detected objects in JSON. It would be great to be able to take this JSON output and have humans add, edit or label the bboxes, which can either be used for finetuning multimodal LLMs or downstream tasks.
Describe the solution you'd like
Currently images are displayed using
image_to_html
to encode images as html in aTextField
. We can create aImageField
to contain an image url and other metadata (width, height, scale, offsets), and aImageSpanQuestion
where users can createRectangleLabels
andPointLabels
stored as responses or suggestions.Describe alternatives you've considered
There isn't really a workaround to annotating region of interests without drawing on top of the image.
The text was updated successfully, but these errors were encountered: