Labeling new class

[karthyyy]

Search before asking

• I have searched the Autodistill issues and found no similar feature requests.

Question

Hi,

Whether Autodistill can be used for labeling new classes/objects which are not common like sockets, ICs etc ? I tried using the same but there is no images in the folder "images_labeled".

Regards
Karthik

Additional

No response

[Jordan-Pierce]

@karthyyy you should play around with Grounding-SAM and Grounding-DINO on a huggingspace demo, that way you get an idea if you're classes can be located and what threshold values you should use before running the entire script.

[karthyyy]

@Jordan-Pierce

Thanks for the quick reply. Sure I will try it out.

Is it like, there are a predefined set of classes in the Grounding-DINO model ? In that case, how to get those details ?

Regards
Karthik

[lab176344]

@karthyyy Unfortunately the paper does not give any info about the training data. But the model is evaluated on COCO, LVISBenchmark and ODinW. if your classes are present in one of these datasets, there is a chance you might get something

[Jordan-Pierce]

@karthyyy also, many of these one-shot detection / instance segmentation foundational models will detect "things" in general. So depending on your class categories, you can always toggle the thresholds (text, box, and IoU) to get hits.

https://huggingface.co/spaces/linfanluntan/Grounded-SAM

[Davisonyeas]

I get an empty output when I use autodistill. I suspect that it is because my prompt class is unknown to the base model training data. The question now is: How can I use autodistill on my custom data that contains custom classes?

[lab176344]

@Davisonyeas please refer to the comment by @Jordan-Pierce, with autodistill the base model uses the default thresholds you can play around with thresholds in the HF spaces and add those arguments in your base model

[Davisonyeas]

@lab176344 I have tested with the hf spaces thresholds, and it recognized my class.
Question now is: What are the arguments for threshold in autodistill?
I have used threshold and conf, but both are not arguments
Error message is: TypeError: DetectionBaseModel.label() got an unexpected keyword argument 'threshold'

[Davisonyeas]

I have viewed the detection_base_model.py file, and neither confidence nor threshold are arguments in the label function
So how do I set my confidence threshold?

[lab176344]

I think currently you cannot pass the parameters directly when initialising the base model. One option might be you can modify it here https://github.com/autodistill/autodistill-grounding-dino/blob/c8991645a75ddb0d4f93a5a96968f0a08f1faef6/autodistill_grounding_dino/helpers.py#L71 if you are using Grounding DINO. I don't a clean way to do it.

[Jordan-Pierce]

I think currently you cannot pass the parameters directly when initialising the base model. One option might be you can modify it here https://github.com/autodistill/autodistill-grounding-dino/blob/c8991645a75ddb0d4f93a5a96968f0a08f1faef6/autodistill_grounding_dino/helpers.py#L71 if you are using Grounding DINO. I don't a clean way to do it.

No. @Davisonyeas you're looking at the template class for all potential base models. Go to AutoDistill's repo and look at the repo that corresponds to the base model you want to use, and look at the input parameters when initializing:

Grounded-SAM

# Line 29

class GroundedSAM(DetectionBaseModel):
    ontology: CaptionOntology
    grounding_dino_model: Model
    sam_predictor: SamPredictor
    box_threshold: float
    text_threshold: float

    def __init__(
        self, ontology: CaptionOntology, box_threshold=0.35, text_threshold=0.25
    ):
        self.ontology = ontology
        self.grounding_dino_model = load_grounding_dino()
        self.sam_predictor = load_SAM()
        self.box_threshold = box_threshold
        self.text_threshold = text_threshold

...

Each base model might have different input parameters; Dino and SAM both allow for threshold values, but not all other base models do.

[lab176344]

Sorry I overlooked code @Jordan-Pierce I was checking the load dino part my bad =(

[Jordan-Pierce]

Sorry I overlooked code @Jordan-Pierce I was checking the load dino part my bad =(

No worries, I did the same exact thing. Cheers.