cosine_similarity not defined

[jakubhejhal]

Search before asking

• I have searched the Autodistill issues and found no similar bug report.

Bug

I want to use composed model:

    model = ComposedDetectionModel(
        detection_model=GroundingDINO(
            CaptionOntology({dino_prompt: dino_prompt}),
            box_threshold=box_threshold
        ),

        classification_model=CLIP(
            EmbeddingOntologyImage(classes_to_images)
        )
    )

But I'm getting:

      1 def predict_and_plot(img_path: str, base_model):
----> 2     results = base_model.predict(img_path)
      3     plot(
      4             image=cv2.imread(img_path),
      5             classes=base_model.ontology.classes(),
      6             detections=results
      7     )

File ~/work/.venv/lib/python3.10/site-packages/autodistill/core/composed_detection_model.py:76, in ComposedDetectionModel.predict(self, image)
     73 # save as tempfile
     74 region.save("temp.jpeg")
---> 76 result = self.classification_model.predict("temp.jpeg")
     78 if len(result.class_id) == 0:
     79     continue

File ~/work/.venv/lib/python3.10/site-packages/autodistill_clip/clip_model.py:74, in CLIP.predict(self, input)
     71     with torch.no_grad():
     72         image_features = self.clip_model.encode_image(image)
---> 74         return compare_embeddings(
     75             image_features.cpu().numpy(), self.ontology.embeddingMap.values()
     76         )
     77 else:
     78     labels = self.ontology.prompts()

File ~/work/.venv/lib/python3.10/site-packages/autodistill/core/embedding_ontology.py:38, in compare_embeddings(image_embedding, comparison_embeddings, distance_metric)
     34     comparisons = []
     36     for comparison_embedding in comparison_embeddings:
     37         comparisons.append(
---> 38             cosine_similarity(
     39                 image_embedding.reshape(1, -1), comparison_embedding.reshape(1, -1)
     40             ).flatten()
     41         )
     43     return sv.Classifications(
     44         class_id=np.array([i for i in range(len(comparisons))]),
     45         confidence=np.array(comparisons).flatten(),
     46     )
     47 else:

NameError: name 'cosine_similarity' is not defined

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[jakubhejhal]

I fixed it by changing the embedding_ontology.py in the following way:

def cosine_similarity(a: np.array, b: np.array) -> np.array:
    """
    Calculate the cosine similarity between two vectors.

    Args:
        a: The first vector.
        b: The second vector.

    Returns:
        The cosine similarity between the two vectors.
    """
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

def compare_embeddings(
    image_embedding: np.array,
    comparison_embeddings: List[np.array],
    distance_metric="cosine",
):
    """
    Calculate the similarity between an image embedding and all embeddings in a list.

    Args:
        image_embedding: The embedding of the image to compare.
        comparison_embeddings: A list of embeddings to compare against.
        distance_metric: The distance metric to use. Currently only supports "cosine".

    Returns:
        A list of similarity scores.
    """
    if distance_metric == "cosine":
        comparisons = []

        for comparison_embedding in comparison_embeddings:
            comparisons.append(
                cosine_similarity(
                    image_embedding.reshape(-1), comparison_embedding.reshape(-1)
                )
            )

        return sv.Classifications(
            class_id=np.array([i for i in range(len(comparisons))]),
            confidence=np.array(comparisons).flatten(),
        )
    else:
        raise NotImplementedError(
            f"Distance metric {distance_metric} is not supported."
        )
....

[capjamesg]

Once you have accepted the CLA, we will merge your fix in #138. I am closing this issue so we can continue correspondence on the PR.