-
Notifications
You must be signed in to change notification settings - Fork 4.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question]: Approaches to searching documents requiring different kind of embeddings (multimodal too) #13594
Comments
To effectively search over a knowledge base with documents requiring different kinds of embeddings, including multimodal ones, LlamaIndex offers a robust approach that aligns well with the options you're considering. Here's a concise overview tailored to your needs:
For implementing these strategies, consider leveraging vector space models for semantic representation, utilizing specialized embeddings for code and text, and integrating with language models for enhanced query processing. Customizable ingestion pipelines and efficient indexing mechanisms are also crucial for handling large and diverse datasets. For more detailed guidance and examples, you might want to explore the source code and documentation provided by LlamaIndex, which offers insights into the implementation of these strategies.
|
Basically it's 2. Create indexes for each type. The multimodal index is one specific use case of these, for images and text, but the general approach makes sense, since each modality requires a specific embedding model |
the response from dosubot is useful, but nothing beats a human. Thank you @logan-markewich ! |
Question Validation
Question
Hello,
I am reflecting on how to tackle the problem of searching over a knowledge base that could require different modalities for embeddings. To make this more concrete, my users are used to upload:
I recall llama_index has some approaches to images and tables, but is there a general preferred pattern to do this?
In general, I see some options to handle this:
Do you see more pattern to apply? Which would be the "llama-index" way of approaching the problem, if any?
The text was updated successfully, but these errors were encountered: