Return matches above a similarity score?

[fgreco55]

Hi... I'm progressing on using Milvus in my course materials (using Java). I have a question about filtering the responses based on the similarity score. I can easily return the (at most) top N matches. Still, I need to return the (at most) top N matches with similarity scores above a threshold (eg, .80). I know I can filter on a column, but I couldn't find info on how to filter on the similarity score.

I certainly can do this manually, but I wanted to know if Milvus had this capability inherently.

Thanks,
Frank

[fgreco55]

I discovered withParms() and now trying to decipher the docs on what "radius of a search circle" means...

I'm good for now, but just a suggestion to have someone review your docs for clarity.

[fgreco55]

What is the specific meaning of "radius" and "range_filter"? Are those specific vector db terms?
Why not just call them "range_min" and "range_max"?
How would I return results that are within a percentage, eg, "give me results that are 90% similar to my embedding vector"

[xiaofan-luan]

1. there is no way to evaluate the percentage of distance, unless you calculate distance for all vectors.
2. radius and rangefilter are differentt implementations. radius use index to accelerate, while range filter is nothing but a filter to filter out very similar result. so they are very different concept. This is to avoid user misuse like range_min and range_max.

[yhmo]

For IP/COSINE metric, distance/similarity range is [1.0, -1.0], 1.0 is the most similar item.
For L2 metric, distance range is [0.0, +∞], 0.0 is the most similar item.

"radius" is equal to the "range_max", "range_filter" is equal to "range_min".
This graph shows how they work for range filtering:

[yhmo]

"give me results that are 90% similar to my embedding vector" ------------------ I think it is hard to determine the meaning of "90% similar".

For L2 metric, how do you define similarity by percent? The distance range is unlimited: [0.0, +∞], then what is "90% similar"? From 0.0 to 1000? Or 0.0 to 99999?

For IP/COSINE metric, yes the range is limited, can we say [0.8, 1.0] is "90% similar"?

[fgreco55]

Thanks for the reply and explanation.