A Webapp that uses Retrieval Augmented Generation (RAG) and Large Language Models to interact with a PDF directly in the browser.
There are three main pillars this tool is based on:
Ask my PDF uses PDF.js to process a PDF locally and split the content up into separate lines.
Every line will be mapped to a 384 dimensional dense vector space using all-MiniLM-L6-v2 with TransformersJS. Those entries are then stored in an in-memory VectorDB directly in the browser.
As soon as a query is submitted, it is also vectorized and the cosine similarity search is used to find the most similar text sections, together with the lines surrounding them, so as not to lose the context.
The text sections found this way together with the query and a few instructions are then used as the input prompt to the Gemma-2B LLM, compiled to WebAssembly and WebGPU using MLC LLM, which will then generate a response.