This repository has been archived by the owner on Aug 2, 2022. It is now read-only.
Got different results depending on what shard request is landing #342
Assignees
Labels
question
Further information is requested
Comments
|
Hi @Hronom, Taking a look. Will update this thread when I find something. |
|
So, for search, ES will send the request to 6 shards (could be either primary or replica) depending on routing. Then, like you said in the expected behavior, each segment will return k results and then each shard will return top size results and ES will return the top size results from those. One potential difference is that the primaries/replicas have different number of segments. Could you run Another potential difference is that if you are indexing with replicas enabled, the replicas will build their own segments' graphs and these graphs will be different from the primaries. To make it so that the replicas have the same graphs as the primaries, you could disable replicas when indexing, and then enable them after indexing workload finishes. This will copy the primaries to the replicas so that they do not have different graphs. However, to be able to do this depends on your workflow. If you are continuously updating the index, this probably won't work. A few other questions:
|
Also we use post filter in our query. So this is why we set |
|
Also how difference in segments affects results? During indexation we do routing and use custom id, so all documents places on specific shards. |
|
The total hits returned is the number of results returned for all of the segments. Assume that a replica shard has 4 segments (each has a graph) with 25,000 documents each, and a primary has 2 segments with 50,000 documents each. If k=30, searching the replica will return 120 results, while searching the primary would return only 60 results (30 from each). If size=3, the top 3 results would be returned from the primary or replica. I am curious what the number of segments are for your primary and replica shards. Could you provide that information? Also, how are you measuring the quality of the search (i.e. recall)? Is there a noticeable performance drop in quality of the search when using one routing compared to another? |
This is not done yet, we done only offline evaluation(out from Elasticsearch) to confirm that vector space provides good results. Also our index not static, we receive updates continuously |
|
Hi @Hronom Could you provide output of I am curious if the difference in hit size will impact the recall significantly, with the post filter enabled. If it does, you could always increase Jack |
|
Hello @jmazanec15 Increasing of |
|
@Hronom For search, the request will go to 6 shards. Routing will determine which shard (for example one search may go to the primary, the other may go to the replica). Looking at shard 0, the replica has 38 segments, while the primary has 22 segments. During search, each segment will be searched sequentially and return The post-filter will filter out those hits and return the top 3. To control the number of segments, you can periodically forcemerge. To increase the accuracy of the search per segment, you can increase m, ef_construction or ef_search. However, before making any changes, I would first evaluate the recall with and without routing enabled. I do not think it should be impacted, but if it is, you can tweak those parameters above |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
We get different results depending on what shard request is landing.
We have:
6shards,1replica100dimensions in vectorsR20210201-P2sizeset to3kset to30If we don't use
routingour results are jumping depending on what shard request is landing.The expected behavior:
we get top k from each segment -> merge results from nodes by score on node that receives requests -> get top 3 from merged list -> top 3 goes in response.
Seems like a bug.
Please can you explain: what under the hood happens that leads to the different behavior?
The text was updated successfully, but these errors were encountered: