@Snippets

[RXminuS]

This is a PR to include experimental @snippet support.

These snippets can be inserted just like normal context

The content of the snippets is inserted into the conversation so that they can tweak the responses by the LLM.

But also do cool things like:

TODO:

• Prompt tuning
• Content Security Policy verificaiton
• In-conversation injectionPoint support
  • Multi injection mixins
• Fix remaining TODO's

Test Plan:

Behind experimental flag, unit tests pass for existing code

[PriNova]

Hey @RXminuS ,
I like the idea to have the ability of choosing pre-defined pre-instruction prompts for the chat, but also for edit commands. Switching between via '#tag' seems reasonable.
If it can be customized via .cody config file, based on User- or Workspace-Profile will be amazing.

[PriNova]

Hey @RXminuS I saw that you included mermaid mixin feature. This is epic.

One recommendation I can give is, that internally Cody should fetch the mermaid API for the flow chart and sequence diagram as context to correctly provide the mermaid syntax. In complex use cases it often fails.
As advice for content fetching use:

Flow Chart
Sequence Diagram

[RXminuS]

One recommendation I can give is, that internally Cody should fetch the mermaid API for the flow chart and sequence diagram as context to correctly provide the mermaid syntax. In complex use cases it often fails. As advice for content fetching use:

Yeah that's a good idea! I'll have to rework it a little so that I can mix-in at multiple points but then that's completely doable. I think that would improve responses a lot!

[RXminuS]

@PriNova I added a bit hacky way of doing this but it seems to work, so maybe good enough for now? Mermaid is still quite tedious with whitespace so graphs do still fail at times.

[PriNova]

@PriNova I added a bit hacky way of doing this but it seems to work, so maybe good enough for now? Mermaid is still quite tedious with whitespace so graphs do still fail at times.

This looks awesome. Did you try fetching the API docs like I mentioned above?
If the user specifies the type of graph via @snippet:visual-Flow, @snippet:visual-Pie or @snippet:visual-Sequence, etc, then retrieving the appropriate API should be doable.

What most newer LLMs also understand are GraphViz dot files.

[RXminuS]

This looks awesome. Did you try fetching the API docs like I mentioned above?

Yeah I did but they waste a lot of tokens doing both the mermaid graph themselves and then the exact same block as code. That's why I just quickly made a gist that combines all the docs/cheatsheets and removes the duplicates. Not a long term solution but having the examples seemed to improve results by quite a bit.

I think the biggest amount of mistakes comes from how there's a high likelyhood of some extra whitespace making it in (and breaking the syntax) probably due to tokens with trailing whitespace being more commonly used than ones with explicit separate whitespace. This has been an issue I've had before trying to generate structured data.

If the user specifies the type of graph via @snippet:visual-Flow, @snippet:visual-Pie or @snippet:visual-Sequence, etc, then retrieving the appropriate API should be doable.

It feels a bit too verbose IMHO? I would like it to just pick the right graph based on my question.

[PriNova]

Yeah I did but they waste a lot of tokens doing both the mermaid graph themselves and then the exact same block as code. That's why I just quickly made a gist that combines all the docs/cheatsheets and removes the duplicates. Not a long term solution but having the examples seemed to improve results by quite a bit.

You are right, it takes around 5k tokens for the API of one chart-type. That's a lot.

I think the biggest amount of mistakes comes from how there's a high likelyhood of some extra whitespace making it in (and breaking the syntax) probably due to tokens with trailing whitespace being more commonly used than ones with explicit separate whitespace. This has been an issue I've had before trying to generate structured data.

The most common chart types are simply formatted, which can be done as a post-formatting step. The first line has no whitespaces and all following lines are prepended with whitespace. Maybe this could be a work-around.

[abeatrix]

love the new @-mention menu! I haven't conducted a full review yet but left a question/feedback regarding how we inject the @snippet in preamble instead of chat message.

Also, it'd be great to add a new e2e test cover the new menu for @snippets 😄

[abeatrix]

Hmmm do we want to alter the preamble instead of the last human message text for @-mention snippets ?
This seems to make it hard to debug using the exported transcript because the prompt for the snippet doesn't show up in the transcript history, and this might not work when the chat transcript gets too long?

[RXminuS]

Yeah I wasn't sure about the correct injection point so wanted to implement both. My main reasoning was to keep it similar to the normal instructions config option but just "ad hoc" which is why I started there.

But I think both of your observations are good arguments to change it. 👍