Use with frequent updates

[z0mb1ek]

Hi! Big thanks for this project, it is great!
Please tell me if I understand correctly: If I often add vectors or often update vectors, I will have many segments and search performance will be slow?

[jmazanec15]

Hi @z0mb1ek ,

Yes that is correct. How much slower depends on how many segments there are. The hnsw search complexity scales O(log(n)), where n is the number of vectors (1).

Lucene will search the segments sequentially. So searching 5 segments with 100,000 vectors would take 5 *log(100,000) ~= 83. Searching 1 segment with 500,000 vectors would take log(500,000) ~= 19.

[z0mb1ek]

thx @jmazanec15

Are there any options for update at least once every 2-3 hours? Without downtime

[jmazanec15]

What do you mean by update?

Regardless, you won't get downtime.

[z0mb1ek]

add portion of vectors, delete them