You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As chat histories get longer, passing the entire history on every call is not a good practice. More so, user expects information from several messages ago to be available as context.
Goal: Improve size of the chat history context window to allow users to reference messages that fall outside existing window.
Solution: Leverage semantic search to index the entire chat history of a conversation and pull messages that are related to the latest message from the user.
Implementation:
Create a pseudo-Pipeline object that uses a custom source connector that simply bypasses messages written to it into a vector database.
Pipeline is declared with an Embed Connector and Sink Connector to be used as part of the operation.
At search we would run a normal search against the sink with filters to only pull messages from the given conversation.
Then the user would add the retrieved messages as context into the conversation alongside the last 3-4 messages
As chat histories get longer, passing the entire history on every call is not a good practice. More so, user expects information from several messages ago to be available as context.
Goal: Improve size of the chat history context window to allow users to reference messages that fall outside existing window.
Solution: Leverage semantic search to index the entire chat history of a conversation and pull messages that are related to the latest message from the user.
Implementation:
Pipeline
object that uses a custom source connector that simply bypasses messages written to it into a vector database.search
we would run a normal search against the sink with filters to only pull messages from the given conversation.Prototyped: https://github.com/NeumTry/Pensieve
Other ideas:
The text was updated successfully, but these errors were encountered: