Is it normal for 10,000 512-dimensional vectors to take more than 300ms for a single query?

[secsilm]

I recently listened to a case study from your team, which mentioned that they have millions of vectors (dimension unknown), using the FLAT index, and the average query time for the top 1000 is only 41ms. Their machine configuration is a cluster of 8 machines with 16 cores and 64GB of memory.

However, we only have around 10,000 512-dimensional vectors, also using the FLAT index, and the query takes more than 300ms (using the Strong consistency level). Our machine is a single machine with 12 cores and 64GB of memory, with an Intel(R) Core(TM) i5-10600 CPU @ 3.30GHz.

I have two questions:

1. Is our time consumption normal?
2. Although I specified a consistency level when creating the collection, I see that different consistency levels can be used for searching on attu. How can I achieve this in my code (python)?

Thank you!

[yhmo]

Milvus maintains data consistency by timetick machinery, the root coordinator sends a timetick message to Pulsar for each 200ms, if all the other nodes(querynodes, datanodes) consume a timetick message, then we know the data before this time point is already ready for search.
https://milvus.io/docs/time_sync.md#Time-Synchronization

Consistency level affects the data visibility for search requests.
https://milvus.io/docs/consistency.md#Consistency-levels

• Strong, wait until a recent timetick message is consumed by all the nodes to make sure the data before "now" is visible, this will add extra 200ms ~ 400ms to search latency
• Bounded, allows data invisible within a time window, in normal cases, it doesn't wait for any timetick messages. In intensive insertions, it might wait and add extra search latency.
• Eventually, always execute search requests without waiting any timetick messages, it is fastest but some recent data might be invisible to search.

Change the consistency level to Bounder or Eventually then you will get fast query.

In python, set the level like this:

search_param = {
        "expr": expr,
        "data": search_vectors,
        "anns_field": VECTOR_FIELD_NAME,
        "param": {"metric_type": "L2", "params": {}},
        "limit": 5,
        "consistency_level": "Eventually",
    }
results = collection.search(**search_param)

[secsilm]

And is the time consumption of client.get around 200-300ms (one id) normal? Is it related to consistency_level?

[yhmo]

client.get() calls collection.query() to fetch data. The collection.query() also has a parameter "consistency_level".
So you can pass the consistency_level like this:

client.get(
  collection_name="xxx",
  ids=[1],
  consistency_level="Eventually",
)

[secsilm]

OK, it worked. Thanks.

[xiaofan-luan]

the reason is you are using Strong consistency level.
Change to bounded staleness or eventual will help. with strong consistency you will expect to get 200ms latency overhead