This repository is a toolkit for using the Segment Anything Model (SAM) developed by Meta AI. It is capable of accurately "cutting out" any object from an image with just a single click.
SAM is a promptable segmentation system that exhibits zero-shot generalization to unfamiliar objects and images, eliminating the need for additional training. By providing prompts that specify what needs to be segmented in an image, SAM can perform a wide range of segmentation tasks without requiring additional training.
- Easy inference with SAM, supporting bounding boxes, points, masks, and text prompts
- Automatic mask generation
- Finetune SAM on custom datasets
- GroundingDINO integration for text prompt segmentation
- Training a custom decoder to auto segment a specific type of object
- Prompt-Tuning SAM: Training a prompt embedding that segments a specific type of object
First, install groundingdino from its repository, this is a dependency for segmate:
NOTE: There is an issue with the setup script in the GroundingDINO repository causing it not able to install torch properly, please manually install PyTorch for now. For other issues, refer to the installation guide:
pip install -U git+https://github.com/IDEA-Research/GroundingDINO.gitThen, install segmate from PyPI:
pip install segmateTo use the provided code snippets, follow the steps below:
- Import the required modules and initialize the necessary objects:
import torch
from segmate.segmenter import SAM
from segmate.object_detector import GroundingDINO
import segmate.utils as utils
# Model checkpoint path for GroundingDINO is optional. If no path provided, it will download from HuggingFace
od = GroundingDINO(device='cuda', ckpt_path='PATH_TO_CHECKPOINT')
sm = SAM(model_type='MODEL_TYPE', checkpoint='PATH_to_CHECKPOINT', device='cuda')- Perform segmentation with bounding box prompts:
masks = sm.segment(image=input_image, boxes_prompt=bbox)
utils.show_masks(image, masks)Detailed package documentation: SegMate Docs
If you have any questions or need assistance, please don't hesitate to reach out to our support team or join our community forum. We hope you find this toolkit valuable and look forward to seeing the incredible applications you create with SAM!
The code in this repository is published under 3-Clause BSD license (see LICENSE file).