Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE_REQUEST] Support for single-space indentations with markdown #2176

Open
SSJGabraham opened this issue May 2, 2024 · 6 comments
Open
Labels
🦄 Feature Request [ISSUE] Suggestion for new feature, update or change

Comments

@SSJGabraham
Copy link

SSJGabraham commented May 2, 2024

Have you searched for similar requests?

Yes

Is your feature request related to a problem? If so, please describe.

No response

Describe the solution you'd like

Add the ability for single-line indentations with bullets (and anything that can be indented in the same way) to be recognized by the UI and rendered properly.

Example

This text:

- bullet 1
 - bullet 2
   - bullet 3

should result in:

  • bullet 1
    • bullet 2
      • bullet 3

At the moment, it displays like this:

  • bullet 1
  • bullet 2
  • bullet 3

Describe alternatives you've considered

Naturally it's not so much of an issue if the user is utilizing markdown as they can just use the supported 4-space indentation, but the AI itself might decide to use single-line indentations all on its own with the intent of them rendering as desired above, likely from being fed data where this version of markdown is used, so the result is that the intended indentations are not shown and the data presented is not organized as the AI intended.

Additional context

I spoke with @LenAnderson regarding this issue who recommended including these two lines in script.js:
image

However, he recently pointed out that this screws up the greeting page:
It does not screw up the greeting page, I misunderstood.

If this change cannot be filtered in some way to only apply to certain text, this might have destructive ramifications in many parts of the UI, so it would likely need to be tested thoroughly.

Priority

Low (Nice-to-have)

Are you willing to test this on staging/unstable branch if this is implemented?

Yes

@SSJGabraham SSJGabraham added the 🦄 Feature Request [ISSUE] Suggestion for new feature, update or change label May 2, 2024
@Wolfsblvt
Copy link
Contributor

While I think markdown nested lists would be nice to have, I'll throw in a quick fix for the time being.
Dunno if the single-space nesting is that easy to use.

Use a custom regex replace on AI output (maybe even user input), and replace it with four times the spaces.
The following works for nested levels 1-3, which should be enough.

Regex: /^( {1,3})-/gm
Replace: $1$1$1$1-

See it in action here: https://regex101.com/r/VrrhEg/1

@LenAnderson
Copy link
Contributor

However, he recently pointed out that this screws up the greeting page:

Huh?
Turning off 4-space nesting in showdown does not mess anything up in the greeting message. And I'm not sure what that screenshot has to do with it.

Outside of an actual chat you don't get proper, if any, markdown support. That has nothing to do with this switch.

@Wolfsblvt

While I think markdown nested lists would be nice to have, I'll throw in a quick fix for the time being.
Dunno if the single-space nesting is that easy to use.

We already have nested list, just with 4 instead of the commonly used 2 spaces. The showdown option simply enables using 2 spaces (not single spaces) like probably 99% of markdown converters. I don't see how that would make it difficult to use.
2-space nesting is what most people somewhat familiar with markdown should be accustomed to. And it seems to be (anecdotally) how many LLMs apply markdown as well.

@SSJGabraham
Copy link
Author

@LenAnderson

Huh? Turning off 4-space nesting in showdown does not mess anything up in the greeting message. And I'm not sure what that screenshot has to do with it.

Outside of an actual chat you don't get proper, if any, markdown support. That has nothing to do with this switch.

We talked about it hours ago after I made this issue, I was confused because you said "markdown does not work correctly on the greeting page" and then posted that screenshot, so I thought you were trying to say that adding those two lines it was screwing formatting up on the greeting page and were showing an example of it. That is good to know that the formatting does not escape the chats, though, as it is less worry that changes there will lead to unexpected weird stuff in the UI.

The showdown option simply enables using 2 spaces (not single spaces) like probably 99% of markdown converters

Are you talking about something other than the disableForced4SpacesIndentedSublists: true lines? Because I did try it briefly and was able to do 1 indent easily with a single space, although trying to do a 2 indent resulted in the third item remaining on 1 indent until I added in 4 extra spaces for a total of 5.

Worth noting, my particular model only tried to go 1 indent deep, and I don't often see them use complicated markdown for formatting, so I can only guess how often more complex formatting even turns up. I have a worry, though, that enabling compatibility for, say, 1-space, 2-space, and 4-space indenting would be difficult to manage when ST has to guess which one you are trying to use.

SSJGabraham added a commit to SSJGabraham/SillyTavern that referenced this issue May 3, 2024
@SSJGabraham
Copy link
Author

SSJGabraham commented May 3, 2024

Tried some testing with this. The model I am using seems to be inconsistent with its usage of markdown. Sometimes it will use 2 spaces. It seems to be comfortable with using any number of between 1-4 spaces to create a sublist, and then if it tries going deeper to create a second sublist, it will either double the number of spaces for the second sublist or do something weird like 3, then 5 spaces like

- Bullet 1
   - Bullet 1.1
     - Bullet 1.1.1

At the very least, whatever it chooses for the first sublist gets recognized just fine. I have yet to see it do 4 spaces (or 1.1), then 8 spaces (or 1.1.1), and its only really if it decides to nest a sublist within a sublist that things get tricky.

@SSJGabraham
Copy link
Author

SSJGabraham commented May 4, 2024

image
The above picture demonstrates how tabs react when disableForced4SpacesIndentedSublists: true is set. Instead of spacing in increments of 1, or 2, bullets are rendered in ranges instead. This will be confusing to anyone who is expecting markdown to behave a certain way, and the alternative is to add behavior in to make it consistent and turn it into an optional setting so it can be toggled which is also not ideal given how many people might actually need or use it. Can maybe place this in an extension of some kind bundled with other formatting customizations including for any other markdown inconsistencies that the user might prefer.

@Cohee1207
Copy link
Member

I'd prefer to go with it YOLO-style and enable it for all in staging, then wait for user feedback. I don't see any big areas of failure here anyway.

The model I am using seems to be inconsistent with its usage of markdown

Model skill issues are luckily not something that ST can or should fix.

@SSJGabraham feel free to PR that change in, as it seems you've been running it for quite some time already and would have reported on further issues (if any).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🦄 Feature Request [ISSUE] Suggestion for new feature, update or change
Projects
None yet
Development

No branches or pull requests

4 participants