Inference time

[sbhttchryy]

Hello, are there any statistics regarding the inference time for the experiments in the 'Open Domain Question Answering over Tales via Dense Retrieval'?

[sbhttchryy]

@eisenjulian

[eisenjulian]

Hi there, we don't have such statistics at the moment. For the paper's experiments, we used a brute-force nearest neighbour search, which in a production application can be dramatically improved by using an index such as FAISS or https://cloud.google.com/vertex-ai/docs/matching-engine

Let us know if you run some benchmarks, I expect the bottleneck would be in extracting the table representation, which should vary depending on the model and architecture (TPU vs GPU vs CPU).