Our tool enables interactive use of segment anything for rapid image segmentation with low RAM requirements (optional bf16 mode).
Demo Video:YouTube
- Semi-automatic Annotation: utilizes SAM with point and bbox prompts.
- Manual Annotation: click or drag to draw polygons (0.15s per point).
- Polygon Adjustments: delete points and adjust object occlusions.
- Polygon Visualization: Preview groups and semantic/instance segmentation masks.
- Supported formats: MSCOCO, YOLO, LabelMe, and VOC (also xml)
For more features, see the Features Description.
There are three ways to install ISAT-SAM:
- from source code (recommended)
- pip install
- from .exe
conda create -n isat_env python=3.8
conda activate isat_env
To use GPU, please install Pytorch-GPU on Windows OS frist.
git clone https://github.com/yatengLG/ISAT_with_segment_anything.git
cd ISAT_with_segment_anything
pip install -r requirements.txt
Download the checkpoint, and save in under: ISAT_with_segment_anything/ISAT/checkpoints
After version 0.0.3, you can manage checkpoints within GUI, click [menubar]-[SAM]-[Model manage] to open the GUI.
Now support SAM, sam-hq, MobileSAM, and EdgeSAM etc.
pretrained checkpoint | memory | size | |
---|---|---|---|
SAM | sam_vit_h_4b8939.pth | 7305M | 2.6G |
sam_vit_l_0b3195.pth | 5855M | 2.6G | |
sam_vit_b_01ec64.pth | 4149M | 375M | |
sam-hq | sam_hq_vit_h.pth | 7393M | 2.6G |
sam_hq_vit_l.pth | 5939M | 1.3G | |
sam_hq_vit_b.pth | 4207M | 379M | |
sam_hq_vit_tiny.pth | 1463M | 43M | |
mobile-sam | mobile_sam.pt | 1375M | 40M |
edge-sam | edge_sam.pth | 960M | 39M |
edge_sam_3x.pth | 960M | 39M | |
sam-med | sam-med2d_b.pth | 1500M | 2.4G |
python main.py
Note that the version may be lower than source code version if installed with pip
conda create -n isat_env python=3.8
conda activate isat_env
To use GPU, please install Pytorch-GPU on Windows OS frist.
pip install isat-sam
isat-sam
The version of exe maybe older than source code.
Download three .zip files, total 2.7G
Download link | |
---|---|
Baidu Netdisk | link:https://pan.baidu.com/s/1vD19PzvIT1QAJrAkSVFfhg code:ISAT |
Click main.exe to run the tool.
The download zip files, container sam_hq_vit_tiny.pth, but somebody told the model not support cpu. You can download mobile_sam.pt to test the tool.
If you want use other models, seeDownload Segment anything pretrained checkpoint
1. Choose the categories in left window of software.
Edit the category in Toolbar-File-Setting.
2. Start annotating
2.1 semi-automatic annotate with SAM.
Click button named [Segment anything] start annotate(shortcut Q).
Click interested area with left button of mouse, click uninterested area with right button of mouse, SAM will calcute masks.
2.2 draw polygons manually.
Click the button [Draw polygon] to start annotation (shortcut C).
Left click to add point into the polygon.
Hold the left click and drag will automaticly add point into the polygon (time interval of 0.15 seconds).
2.3 Undo
Click the button [Backspace] to return to the previous state (shortcut Z).
3. Finish the annotation with [Annotate finished] or shortcut E.
4. Save the annotation with [Save] or shortcut S
1. Modify polygons coordinates
Drag and drop polygon vertices to modify the shape of the polygon.
Drag the polygon to adjust its position.
2. Modify polygons category
Choose the polygon and click [Edit] or double click the polygon, and choose the new category in editing window.
3. Occlusion modification
Choose the polygon and click [To top] (shortcut T) or [To bottom] (shortcut B).
4. Delete polygon
Choose the polygon and click [Delete] to delete the polygon.
1. Preview annotations
Click the [Bit map] to preview semantic and instance annotation masks (shortcut space).
The order of swithing is polygons-semantic-instance.
2. Image window
Click [Zoom in], [Zoom out], [Fit window] (shortcut F) to adjust the zooming distances.
3. Show / hide polygons
Click [Visible] to show / hide polygons (shortcut V).
4. Mask aplha (only effective when using SAM)
Drag the [mask aplha] bar to adjust the mask transparency.
ISAT has a specific format with .json. You can use export it to other formarts.
1. ISAT to VOC
Convert ISAT jsons to PNG images.
2. ISAT to COCO
Convert ISAT jsons to COCO json.
3. ISAT to LABELME
Convert ISAT jsons to LABELME jsons.
4. COCO to ISAT
Convert COCO json to ISAT jsons.
Please support us with a star—it's like a virtual coffee!
... | ||
yatengLG | Alias-z | ... |
@misc{ISAT with segment anything,
title={{ISAT with segment anything}: An Interactive Semi-Automatic Annotation Tool Based on Segment Anything},
url={https://github.com/yatengLG/ISAT_with_segment_anything},
note={Open source software available from https://github.com/yatengLG/ISAT_with_segment_anything},
author={yatengLG, Alias-z and horffmanwang},
year={2023},
}