How to implement conversation?

[unode]

I would like to create some "conversational" plugins.
The core feature currently missing is some form of persistence of state between messages as well as an alternative message routing path for when conversational mode is enabled.

A simple example:

luca: @mbot foodselection tomorrow
mbot: Tomorrow there are 3 available options:

    1. Vegetarian (brought by @mike)
    2. Vegan (brought by @sandy)
    3. Bring your own and share

* Which would you like to have? 

luca: 3
mbot: More options is always better! How would you like it to be displayed to others? 
luca: Gluten-free
mbot: Great!, does "Gluten-free (brought by @luca)" look good to you?
luca: yes
mbot: "Gluten-free (brought by @luca)" is now an option and is available to others.

Has anyone implemented anything of this sort?

I have some ideas on how state could be implemented but the tricky part seems to be the conversation.
Does anyone have suggestions of core features that could make this easier/possible?

[attzonko]

How long do you need the state to be retained? I did some state stuff by just keeping the state in memory, if the bot restarts the state is lost, but for my needs that was fine. The second way I added state retention is by adding a db backend for one of the plugins. In my case I use MySQL, but you can do whatever you want. The bot is in python, so there are plenty of options out there for you to store your state :)

[unode]

I didn't go so much into the technical bits back there.

On my local setup I also have a SQLite/SQLAlchemy db. This could indeed be used for the state and could even be made to persist across reboots.

What I'm not sure about is how to implement the back and forth communication. If you check my example above, the replies by luca don't really follow a pattern that could be matched via @respond_to. I tried this direction for some small cases and while it worked, it was rather fragile. For instance, if two plugins implement the same pattern for a common answer like choosing from a pool of options. Forcing a prefix or suffix for every message also makes the whole interaction less fluid and artificial.

I'm also thinking that this kind of stateful interaction could be made to work through any communication channel but is probably easier to use via private message where you don't need to @bot all the time.

[attzonko]

Yes totally understand this conundrum. In my instance, the best solution I came up with was forcing this type of conversation to happen only as a direct message between the bot and the person. I have not played with the choice stuff much, but I do believe there is some support for displaying a modal with buttons that can correspond to actions. Similar to how some plugins do it. I am pretty sure the API supports it, but I don't think mmpy_bot has everything needed for that.

[unode]

Thanks for the feedback Alex.

Would it be possible for you to share some of the code or the design of the approach you used in your instance?

The only directions I'm seeing right now involve some hacky changes to mmpy_bot's core or some significant ones that would affect the interface between plugins and the messaging classes. Too early to say but some may be backwards incompatible.
Would you still be interested in this and if so, do you have any preference in terms of how to contribute them to mmpy_bot? I.e. would you prefer one large pull request or that I'd fork now and we check later how/if to merge?

[attzonko]

Sure, so I used it for onboarding new users. Here is the function used to start the onboarding:

@respond_to('start foobar onboarding')
def foobar_start_onboarding(message):
    """ ***Direct Message Only!*** Start the **one time** process of getting authenticated"""
    if not message.is_direct_message():
        message.reply("I am sorry for security reasons you must start the onboarding process from a Direct Message.")
        return

    message.send("Visit the [FooBar Token](https://foobar.example.com/) page to create a new foobar token or get an existing token ")
    message.send("If you need help with creating the token take a look at the instructions located [here](https://foobar.example.com/token_help)")
    message.send("According to IT it should take around 10 minutes for the token to become active, so wait 10 minutes after you get the token")
    message.send("Once you have the token and have waited 10 minutes let me know what it is by saying `my foobar token is <your_token>`")

And here is a function used to check if the user has already gone through the onboarding process. The api_keys is a simple 50 line module used to handle API keys stored encrypted in a MySQL database. It has a very simple API (add_key(), remove_key(), get_key(), has_key())

def foobar_onboarding_done(message):
    """ Check to make sure the onboarding process for FOOBAR is complete and return token """
    if not api_keys.has_key(message.get_user_mail(),"foobar"):
        message.reply("this appears to be the first time you have tried to use me to interact with FOOBAR. Please start a Direct Message with me and type `start foobar onboarding`")
        return False
    return True

And here is the function I use to finish the onboarding process:

@respond_to('my foobar token is (.*)')
def foobar_finish_onboarding(message, token):
    """ ***Direct Message Only!*** Provide a token to use when interacting with FOOBAR"""

    if not message.is_direct_message():
        message.reply("I am sorry for security reasons you must use Direct Message when providing your FOOBAR Token.")
        return

    if len(token) > 64:
        message.send("The token you sent, seems too long to be valid, please try again")
        return

    # Check to make sure the token works
    message.send("Hold on, checking your token, this should not take long ....")
    id = foobar_get_widget_by_name(message.get_username(), token)
    if id != "8675309":
        message.send("I was not able to use your token to interact with FOOBAR. Please report this in ~support")

    # Store token for Future use
    api_keys.add_key(message.get_user_mail(),"foobar", token)
    message.send("All done, token stored succesfully. I should be able to get FOOBAR information for you now.")

So then during normal interaction with the bot where it is asked to do things with FOOBAR, all I do is check to make sure the user has been onboarded in the past by adding this check in the respond function:

   if not foobar_onboarding_done(message):
        return resp

That is about it, obviously, I redacted this a bit but you should be able to get the idea of what I did. Let me know if you have any questions.

[unode]

Hi Alex, thanks a lot for the code examples.
If I understand correctly this is a two step process where communication is still done via the respond_to decorator. I misunderstood back there and thought you had something closer to luca's interaction in my example above.

I'll explore what changes I'll have to do to mmpy_bot core for this and get back to you. I'll keep any changes in my fork and we can see how/if to integrate them later.
Thanks again for all the feedback.

[sadok-f]

Hi @unode,

did you find a way how to implement a conversation using mmpy_bot?
I have a similar use case as you do, will be interesting to see if you have any suggestion.

Thanks,

[unode]

Hi @sadok-f

not yet. Time constraints...
Currently only scattered thoughts but happy to bump ideas.

Cheers

[jneeven]

With the refactor in #153, perhaps a nice way to do this would be to register listeners on the fly (currently not supported, but should not be a huge change). You'd have some function with listen_to, which then sends a message back, and then registers a listener for that specific message, such that if you reply to it, that function will be triggered. Since plugins are now classes, keeping track of some kind of conversation state for each user should be very straightforward. I think conversation would be a nice feature to have (and similarly, having multi-stage interactive dialog forms), though it doesn't have a high enough priority for myself to make time for it in the foreseeable future, but I'd be happy to provide some guidance in implementing a framework for this purpose (something along the lines of a start_conversation decorator, Plugin.reply_conversation function, etc)!

[unode]

@jneeven Thanks for your input (and the mention of #153 which looks rather impressive).
What you mentioned above is quite in line with what I tried but didn't manage to finish.
I'll need to bring myself up to speed with the new changes but looks promising.

[attzonko]

@unode Did you ever get what you needed with 2.x on this request?

[unode]

@attzonko I haven't been able to experiment with it yet.
I still plan to but if anyone else wants to take this please go ahead.

[unode]

Just giving a brief update here.

I don't yet have an implementation that is solid enough to upstream but my latest approach, that is hacky but somewhat functional, is to use Mattermost threads as "state" of conversations. By using threads, the bot can then work stateless and retrieve the full context by requesting the full history of the thread to mattermost.

To better support this use-case we would need to have a way to register that replies to a given thread are to be handled by a specific function.

I explored two possibilities:

1. Using interactive messages or dialogs
2. Persisting state in mmpy_bot (e.g. keeping track of the thread_id).

The first works well for pre-defined interaction workflows such as the one described in the opening message of this issue.
Each reply can be a button that triggers the corresponding webhook function. This is already supported by mattermost and mmpy_bot. Dialogs use a verbose syntax but they work.

However the dialog interaction doesn't scale to human-like text interactions that are now possible via large language models (LLM).
For LLMs, the only option I was able to implement was 2. State information was kept in memory as: thread_id -> function_handler.

See below an example interaction. Assume listen_to("^gpt +(.*)", use_conversation=True) is used to decorate the handler function.

John: Hi everyone, how can I add 1 + 1?
Marc: gpt can you answer John's question above?
mmpy_bot: You can add 1 + 1 using concatenation. In this case the result of concatenating 1 and 1 is 11.
Marc: please interpret the + as mathematical addition.
mmpy_bot: Interpreting + as mathematical addition makes the operation 1 + 1 return 2.

behind the scenes indicated by >:

John: Hi everyone, how can I add 1 + 1?
> thread_id not in mmpy_bot's internal reference so the message is ignored
Marc: gpt can you answer John's question above?
> gpt ... is passed to the function decorated by listen_to("^gpt +(.*)", use_conversation=True)
> before returning the answer to mattermost, mmpy_bot stores the current thread_id and a reference to the function handling the request
>> "thread_id : handler_function" stored in mmpy_bot's session as a key-value pair
> this logic needs to be implemented in `listen_to` so a reference to the handler function can be kept.
mmpy_bot: You can add 1 + 1 using concatenation. In this case the result of concatenating 1 and 1 is 11.
Marc: please interpret the + as mathematical addition.
> before regexps are attempted, check if the current thread_id is in memory, if yes, call the corresponding function
> if necessary the handler function can request the full thread history to mattermost before proceeding, otherwise only the last message is processed.
mmpy_bot: Interpreting + as mathematical addition makes the operation 1 + 1 return 2.

An optional "end of conversation" message can be implemented to delete the thread reference from mmpy_bot.
I also added a cleanup routine to delete thread references older than a configurable period (a timestamp had to be stored along with the function reference for this).

So in short, the hacky part is the need to hide the thread based dispatch logic in listen_to and the overhead that now all messages need to be checked against stored threads.

This implementation also suffers from some context problems.
In the above, if Marc started the sentence using word that regexp-matched another handler, it may be ambiguous if all matching handlers or just the stored handler should be called by mmpy_bot. There may be cases for both. It's not clear how to implement this. Will certainly be a "feature not bug" situation.