[Roadmap] Reply to specific parts of text

[hardikbhatia777]

Why
Users should be able to (quickly) respond to certain parts of the AI response. Plain and simple feature, but massively improves QOL.

Description
Much like ChatGPT, when a user selects a certain part of the conversation/reply, a small bubble should appear where clicking the bubble copies the highlighted text into the message box. I'm aware that a similar menu already exists and opens on right clicking on text, but it might not be completely obvious for many users since most LLM UIs use this floating bubble method for replies/actions over selected text.

And the message box should turn something like this:

Requirements
Change to the message box to accommodate replies in some form, or maybe just copy the highlighted text to the message box. Personally I'd like the first option.

I'm willing to submit a pull request for this feature.

• Implement highlighted text selection functionality.
• Change the message composer box to accommodate replies.

[hardikbhatia777]

This is another way of implementing said feature with some more useful options albeit a bit more nuanced.

[enricoros]

Hi @hardikbhatia777 did you make the second design? Looks really interesting and in the direction we'd love to implement this. Where can I see more of it?

[hardikbhatia777]

@enricoros I didn't, this is Gemini's interface. Glad you like the suggestion 🚀

PS. I'm Seven on Discord :)

[enricoros]

I like Gemini's work on the UI. Of all the UIs it's the most similar to my thinking for Big-AGI. This is a feature I had in mind for a while, and it's half implemented on main.

• OpenAI implements it by inserting a system prompt right after the user prompt, and that kind of behavior really only works for OpenAI (most vendors do not support more than 1 system prompt, especially in the middle of text)
• we'd have to abstract it for other models or disable the feature
• Gemini has interesting and clearly prioritized operations
• I'm trying to bring Beam on a selection level, which would be huge for remixing a paragraph with live preview; it's an easy feature that would just take a couple of days to implement but I don't have that kind of time now, would love for someone else to take it

[hardikbhatia777]

Thanks for your feedback. I'd be happy to take this up over next week.

A small clarification - do you want the bubble over highlighted text like ChatGPT and Gemini have, or do you want this to be integrated within the right click menu which has copy, auto-diagram and speak functionalities?

[enricoros]

Hi Seven, you'll find the UI changes on the main branch half implemented, as I started last week. In the ChatMessage.tsx file there are flags up top to enable the context (right click) menu, or the new Overlay menu, which works on a selection.

I think the overlay is better but has less space, especially on mobile. The overlay can fit the new Reply function and the former context actions (Copy, Speak, draw, diagram), but not much more.

What do you think? I like the Gemini looks best, but many users (especially on Mac) never right click, so the 'overlay' presentation is best. Alternatively, it could be a multi-line menu.

Status of implementation:

• selection overlay framework
• showing the reply text on the Composer
• implementation of the reply-to function (OpenAI injects a system message that you can see in the network logs)
• implementation of "cut" (really useful)
• implementation of other functions (shrink, expand..)

I'll be busy for a few days, so please take a look!!

[hardikbhatia777]

Sounds good. I agree, many users never right click and a Gemini-esque approach would work best.

Thanks for clarifying.

[enricoros]

Fully implemented on unreleased branch.

[enricoros]

@hardikbhatia777 this is how it looks like:

Works great with Anthropic and OpenAI and other models.

[hardikbhatia777]

Looks great! 🚀