Number of threads when indexing (omp_get_max_threads
, omp_set_num_threads
)
#3285
Replies: 2 comments
-
Following a discussion with a colleague, this could be related to faiss relying on OMP for parallelism - which is "process parallelism" and thus only uses as many "physical" cores as available on the machine. Trying to rebuild with MKL to see and use alternative openmp runtime (MKL) to see if anything can be gain. Will update... |
Beta Was this translation helpful? Give feedback.
-
I managed to build FAISS with MKL and I can confirm that now In general, I build on a clean debian 12, installing the required dependencies, most importantly the MKL library. A few references/notes:
If I get a change I may try to do an organized build and a simple performance test (or just run the existing tests) |
Beta Was this translation helpful? Give feedback.
-
I am using faiss (indirectly through langchain) to create embeddings.
When I am creating the index (L2), it seems that only about half of my cpus* are being used. I.e.
faiss.omp_get_max_threads()
I am getting a value which is half of the number of coresUsing
faiss.omp_set_num_threads
to set the number directly, or setting environment variableOMP_NUM_THREADS
does not have any effect when using a value over the half-of-my-cores value.*when I am writing "cpus" my intention is total number of cpu-threads on the machine - that's the number show for CPU(s) when I run
lscpu
, e.g., on my work laptop I get:I assume this is not related to langchain is most of the work is done by faiss itself and it's quite a thin wrapper around the library
Any suggestions / pointers? This is quite frustrating and I am wondering what is going on.
A PS
I am running in jupyter lab. Python 3.10.13 - not sure if it matters...
Beta Was this translation helpful? Give feedback.
All reactions