Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multi-user setup with slack doesn't work with a single agent #1356

Open
deefactorial opened this issue May 8, 2024 · 1 comment
Open

multi-user setup with slack doesn't work with a single agent #1356

deefactorial opened this issue May 8, 2024 · 1 comment

Comments

@deefactorial
Copy link

deefactorial commented May 8, 2024

Describe the bug
A clear and concise description of what the bug is.
In the root readme.md file it mentions the suggested setup for a multi-user setup.

How do I implement MemGPT for multiple users?
The REST API for [MemGPT](https://memgpt.readme.io/reference/api) is flexible and leverages PostgreSQL DB or SQLite for its backend. To implement a multi-user setup, first determine the user_id (either create a UUID or use the user_id from your own database). Then [create an agent](https://memgpt.readme.io/reference/create_agent_api_agents_post), and finally use the agent_id and user_id to post a message or run a command. Internally the following occurs:

a user creates an agent
that agent is "owned" by a user
when the user sends the agent a message, that's stored in a message collection (messages are indexed by user and agent ids)
on the higher-level agents side (not talking about db implementation details), the agent can only see a few messages at a time, but has access to all the messages ever sent between it and the user via the recall memory search functions
the database is multi-user, and the REST endpoints function in a way where user data is not shared

From that implementation suggestion this what I what I heard the suggested implementation is:

  • Listen for app mentions
  • When user mentions slack bot
  • Chat with that specific user using user's version of AI persona
  • Ignore all other messages that don't mention slack bot and other users mention of AI persona

Please correct me if I'm wrong in my assumptions.
From what I understand is that this creates a personalised agent for each user, and only interactions with that user are recorded within it's memory.

There are some problems that I can see with this structure:

  1. You need to create a new agent for each user
  2. The agent only creates persona memories with respect to one user, so when the persona is talking with multiple users it's got a split personality, because the memories it creates regarding it's persona are only specific to that user.
  3. It doesn't have a frame of reference with respect to the channel thread, conversations that occur in the channel thread are ignored, so it doesn't remember conversations or messages that are in the channel thread.
  4. memories that are created with a user in private channel or one on one could be shared when chatting with an agent in broader channel like the #general channel.
  5. There is no contextual awareness of the organization in which the agent is chatting with.

I've done a search for similar issues and found this one.
#668

In order to address these issues here is what I'm thinking should be done in order refactor memGPT to support a multi-tenant group chat environment like slack.

  1. There should be one chat agent for the entire organization
  2. The agent should listen/monitor conversations in all public channels, private channels it was invited to, group channels it was invited to and private one on one DM's it was invited to.
  3. The agent should only respond when mentioned using the @ syntax
  4. when creating memories of conversations it should store metadata about which message derived the memory, Who was the sender of the message and who has access to that message.
  5. when doing retrieval of messages it should filter the messages based on the intended audience of the memory and who has access to those memories. So memories sent in one on one conversations are not shared in the general channel.
  6. The agent should keep an array of user memories for each user in the system, each memory should contain metadata about which message or messages it was derived from and who has access to those memories.
  7. The agent should follow GDRP regulations for the right to be forgotten, if a user is deleted within the system all memories of that user should be purged.
  8. When conversing all memories that are used within the conversation should include references to the original message that was used to create the memory. This way the user can validate the memory contents on their own and should have access to that original message.
  9. The agent should keep an array of channel topic memory banks that it uses to reference the channel history of conversation so it retains contextual awareness of what was talked about within the channel.
  10. The agent should retain an organizational awareness of all the public channels and an overview of the content contained within them, as well as user overview as to easily be able to reference channels and users within the entire slack environment.

I did a brief overview of the code prompts and logic and found that Persona and Human objects appear to be like object storage banks for memories in order to implement a structure suggested above I would need to create an Organization object, refactor the Human object to be an array/map of Human objects, and create a channels/threads array/map of objects to store memories about.

Please let me know if there is any current or planned development in any of these directions. or any suggestions you may have in developing this.

Thank you for your time and your efforts in developing MemGPT, I think the work done here is excellent and I'm looking forward to developing it further.

@sarahwooders
Copy link
Collaborator

@deefactorial we have considered making the "core memory" (the memory held in the LLM's context) more flexible (e.g. a key/value store) rather than having designated human/persona sections, but don't have concrete on the roadmap yet. If this is something you're interested in, I recommend implementing an extension of the CoreMemory object that is more general or includes the sections you want (e.g. an organization).

Once you have a custom memory class, you want to give the agent the ability to modify that custom memory class. You can do that by modifying the core memory related functions (https://github.com/cpacker/MemGPT/blob/main/memgpt/functions/function_sets/base.py#L59) to match your memory class. Once you have the memory editing function, you'll want to create a custom preset that connects your memory functions instead:

system_prompt: "custom_prompt"
functions:
  - "send_message"
  - "pause_heartbeats"
  - "custom_core_memory_append"
  - "custom_core_memory_replace"
  - "conversation_search"
  - "conversation_search_date"
  - "archival_memory_insert"
  - "archival_memory_search"

You can also add other functions like delete_user_memories that clears out memories related to a user -- and you might want to extend the 'Passage' (archival memory rows) class to include metadata about what user a memory corresponds to.

This is a pretty advanced use-case that I'm not sure someone has tried before, but we do want to make the memory "programmable" to developers so feel free to reach out here or on discord and we'd be happy to help you make it work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: To triage
Development

No branches or pull requests

2 participants