Brute force search without an index
In the case of IndexFlat
, the added data is just copied to the index without further encoding or organization.
Therefore, it may be useful to short-circuit the indexing altogether.
This makes it possible to:
-
avoid having two copies of the same data in memory
-
do updated on the data between searches.
Faiss provides low-level functions to do the brute-force search in this context.
The functions take a matrix of database vectors and a matrix of query vectors and return the k-nearest neighbors and their distances.
On CPU, the relevant function is knn_L2sqr
or knn_inner_product
.
Example usage in Python: this gist
On GPU, the relevant function is bruteForceKnn
.
An additional advantage is that it takes both CPU and GPU resident data as input or output.
An example usage in python, from pytorch GPU data is here: test_pytorch_faiss.py
Faiss building blocks: clustering, PCA, quantization
Index IO, cloning and hyper parameter tuning
Threads and asynchronous calls
Inverted list objects and scanners
Indexes that do not fit in RAM
Brute force search without an index
Fast accumulation of PQ and AQ codes (FastScan)
Setting search parameters for one query
Binary hashing index benchmark