Implement vector memory

[jerryjliu]

UPDATE

• Re-jigged vector memory to only supply list of chat messages and not handle any thing having to do with composing messages
• Added BaseComposableMemory and SimpleComposableMemory for better separation of concerns
  • With this class, now we can pass a VectorMemory and ChatMemoryBuffer to a SimpleComposableMemory which will do a simple composition on the messages that are retrieved by all of its memory sources
• Modified vector_memory.ipynb and added new composable_memory.ipynb (added these to docs as well)
• Added these classes to api reference (I think?)

OLDER

was way harder than i originally anticipated, summary of main changes:

general interface stuff:

• added input argument to get function in memory
• added put_messages to memory
• replaced all memory.set(..) calls in the agent with memory.put_messages in terms of updating state
• added BaseChatStoreMemory for all memory modules that depend on a chat store

implemented vector memory:

• track each "node" in a vector index
• each "node" is by default a group of messages, instead of a single message. this is because if we index individual messages, we'll retrieve just the user message or just the assistant message, but not both. you may end up retrieving a user message and an unrelated assistant message leading to hallucinations
• we track the current "message batch" in the chat store, to accumulate user message, then assistant/tool msgs, etc.
• we also track all message ids in a separate collection in the chat store. this is to allow us to perform deletes, which right now consists of deleting messages one by one.
• NOTE: deletes from vector stores are hard, we do some hacks to make delete_ref_doc work

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[logan-markewich]

A little confused on the flow here

The chat store is keeping track of the inserted node ids (effectively maintaining order)? Not 100% sure what override last is for

[nerdai]

so I think override_last is needed since we're updating the "batch" as we go. So if its not the end of the batch, we gotta delete the last entered node, and replace it in favor with the newly updated one... i.e. one that has the latest message.

[nerdai]

The chat_store is acting as a temp buffer (I think you had mentioned before, and I'm just getting caught up with now) to track the current user batch i.e., group.

[jerryjliu]

yeah tbh it's a bit hacky, and we're not using chatstore for its intended purpose. but if we want to batch messages together we need some way of making sure the latest node is 1) always updated, but 2) can still be returned as part of vector search

[nerdai]

see working PR here for removing chat_store as buffer: #13729

[jerryjliu]

hm i guess we don't have delete_all in the vector store yet, so we still need to maintain a list of all the ids

maybe fine for now, but may want to change this later to make vector memory less brittle (right now the chat store is in memory, so if we try to reinitialize the vector memory module there's no easy way to delete old data in vector memory)

[nerdai]

I think we have a clear() method now that should delete all. Tbh, wasn't 100% clear on what this note meant. Are you saying that we only need chat_store to maintain the id's. And, since we have "clear()we can implement this without thechat_store`?

[jerryjliu]

yeah basically. i think we still need the cur_user_msg_key for batching, but we no longer need to have a chat store track all user ids

[nerdai]

Sorry, I'm still hung up on this, but if we do a delete_all on the vector store, wouldn't that remove all past batches in addition to the current running batch? I think we would want to delete the current running batch so that we can start a new one with the incoming user message?

[nerdai]

left this for a future PR, but have started a working branch/PR for this that removes chat_store. I have effectively got it working, but it needs further polishing.

#13729