Brackets highlighter keeps highlighted paranthesis after entering ZLE's Vi insert mode

[Iskustvo]

Steps to reproduce:

• Start the zsh in new terminal emulator, without any configuration

  termite -e 'zsh -dfi'

• Use Vi keybindings with support for block/bar cursor changing depending on ZLE's Vi mode

  bindkey -v
  function change_cursor() { [[ "${KEYMAP}" = vicmd ]] && printf $'\e[2 q' || printf $'\e[6 q' }
  zle -N zle-keymap-select change_cursor

• Source z-sy-h from where it's installed, and use main and brackets highlighter

  source "/usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh"
  ZSH_HIGHLIGHT_HIGHLIGHTERS=(main brackets)

• Write the following code

  echo "$()"

• Go back to NORMAL mode with Esc key and position the cursor on the first opened parenthesis.
  Both parentheses should be highlighted to note that those are one logical pair of open/closed parenthesis.
  
• Press a to enter INSERT mode in between parentheses and start writing some code.
  Expectation: No parenthesis is highlighted as soon as you enter INSERT mode.
  Actual behavior: Opened parenthesis stays highlighted while writing in INSERT mode.
  

Versions:

zsh - 5.9-1
zsh-syntax-highlighting - 0.7.1-1

[danielshahaf]

This isn't specific to viins mode; you can get it in vicmd mode if you use a bar cursor there too.

AFAICT z-sy-h does instruct the terminal to render both parentheses in standout, but the terminal doesn't do so for the parenthesis at the cursor position. So, set ZSH_HIGHLIGHT_STYLES[cursor-matchingbracket] to a different value? Or do you think a change in z-sy-h is required?

Thanks for all the bug reports :-)

[danielshahaf]

AFAICT z-sy-h does instruct the terminal to render both parentheses in standout, but the terminal doesn't do so for the parenthesis at the cursor position. So, set ZSH_HIGHLIGHT_STYLES[cursor-matchingbracket] to a different value? Or do you think a change in z-sy-h is required?

Sorry, that's wrong. The cursor-matchingbracket highlight is specifically for the "other" bracket, the mate of the one at the cursor position.

[Iskustvo]

This isn't specific to viins mode; you can get it in vicmd mode if you use a bar cursor there too.

Yes, you are correct, seems like the bar cursor on its own is enough to confuse it.

Or do you think a change in z-sy-h is required?

Well, I would like to achieve the expected behavior.
At the moment, I see two ways for this:

• z-sy-h could check for the cursor shape when doing the highlight matching and just omit it if the cursor has bar shape.
  • This might not be the preferred behavior for people who are constantly using the bar shaped cursor.
• z-sy-h could expose some kind of function or variable so that users can turn pair matching checks on/off when they want.
  • With that, users can simply turn off the pair matching highlight when entering INSERT mode and turn it on when entering NORMAL mode. Something like:

    function change_cursor()
    {
        if [[ "${KEYMAP}" = vicmd ]]; then
            printf $'\e[2 q'
            ZSH_HIGHLIGHT_STYLES[match_brackets]=1
        else
            printf $'\e[6 q'
            ZSH_HIGHLIGHT_STYLES[match_brackets]=0
        fi
    }

Thanks for all the bug reports :-)

Thank you for triaging/solving them!
Btw, I just noticed that the change_cursor function I wrote in the first comment has the last closing bracket highlighted as error, even though it's valid. Adding ; before it didn't fix the highlight. So, would you like me to open another bug? 😄

[danielshahaf]

Yes, you are correct, seems like the bar cursor on its own is enough to confuse it.

More or less. The cursor-matchingbracket style does not cover the bracket under the cursor, but only the mate of the bracket under the cursor. That's by design. It's not immediately apparent in your first screenshot, with those two identical blue blocks, but in xterm I see one blue block and one white block:

The blue block is because the bracket-level-1 and cursor-matchingbracket styles are both applied:

zsh-syntax-highlighting/highlighters/brackets/brackets-highlighter.zsh

Line 33 in caa749d

: ${ZSH_HIGHLIGHT_STYLES[bracket-level-1]:=fg=blue,bold}

zsh-syntax-highlighting/highlighters/brackets/brackets-highlighter.zsh

Line 38 in caa749d

: ${ZSH_HIGHLIGHT_STYLES[cursor-matchingbracket]:=standout}

The white block is just xterm's default cursor shape and color (possibly modified by my distro).

The upshot of all this is that as soon as you change the cursor shape, you'll find that only one bracket of the pair gets highlighted… but that's not because changing the cursor shape stops z-sy-h from highlighting one half of a pair of brackets; it's because that bracket only seemed to be highlighted at first, because the default cursor shape coincides (in your environment) with the default rendering of cursor-matchingbracket.

---

All that being said, it does seem to be the case that cursor-matchingbracket only makes sense when using a block cursor.

• z-sy-h could check for the cursor shape when doing the highlight matching and just omit it if the cursor has bar shape.

  • This might not be the preferred behavior for people who are constantly using the bar shaped cursor.

Yeah. We don't know the user's preference, so we shouldn't guess, but make things configurable.

• z-sy-h could expose some kind of function or variable so that users can turn pair matching checks on/off when they want.

Exactly. We could invent a way to prevent that style from being added, as you suggest (or a way to remove them after being added and before zle processes them — that's admittedly roundabout, but the memo= metadata idea from earlier today would make this easy); or a way to add a style not only for the undercursor bracket's mate but also for the undercursor bracket itself (similar to the cursor highlighter, but only when the cursor is on a matched bracket).

Expectation: No parenthesis is highlighted as soon as you enter INSERT mode.

No parenthesis at all? Or just neither the one at the cursor nor the mate of the one under the cursor? E.g., on the command line outer $(middle $(inner); :), with the cursor on inner closing parenthesis and in insert mode, what parentheses would be highlighted?

[danielshahaf]

Thank you for triaging/solving them!

You're welcome, and sorry for the confusion above.

Btw, I just noticed that the change_cursor function I wrote in the first comment has the last closing bracket highlighted as error, even though it's valid. Adding ; before it didn't fix the highlight. So, would you like me to open another bug? 😄

Minimal example: { : '[' }. For context, see #817 (comment). If you have anything to add, then yes, on a new ticket, please (or on an existing one if appropriate).

[Iskustvo]

Exactly. We could invent a way to prevent that style from being added, as you suggest (or a way to remove them after being added and before zle processes them — that's admittedly roundabout, but the memo= metadata idea from earlier today would make this easy); or a way to add a style not only for the undercursor bracket's mate but also for the undercursor bracket itself (similar to the cursor highlighter, but only when the cursor is on a matched bracket).

Maybe I'm missing the point, but to me it seems that only user-exposed variable that prevents applying of the cursor-matchingbracket style would actually enable user to achieve desired behavior.

After reading your explanation for how cursor-matchingbracket style works, I realized that the issue is actually happening only because my bar-shaped cursor is positioned right behind the closing parenthesis. If it was block cursor, it would be on top of it. Because of that z-sy-h thinks that it's appropriate to highlight the opening parenthesis. Therefore:

• memo solution wouldn't help because it doesn't detect the shape of the cursor and it's irrelevant if parentheses were highlighted before or not.
• Highlighting both the parentheses (either same or differently) while in INSERT mode near closing parenthesis would look nicer than just one, but I still don't consider this the desired behavior. In my opinion, neither should be highlighted in this case.

Probably one more related thing about this is how cursor-matchingbracket should behave while in VISUAL mode.
Would the user want to have already selected parenthesis change color because cursor is currently positioned on the paired one?

No parenthesis at all? Or just neither the one at the cursor nor the mate of the one under the cursor? E.g., on the command line outer $(middle $(inner); :), with the cursor on inner closing parenthesis and in insert mode, what parentheses would be highlighted?

Sorry for my bad description. I would want all highlighting to remain as it was before, only that cursor-matchingbracket style wasn't applied anywhere, because I'm not actually positioned on any parenthesis when I'm in INSERT mode.

Minimal example: { : '[' }. For context, see #817 (comment). If you have anything to add, then yes, on a new ticket, please (or on an existing one if appropriate).

Nah, it's unrelated to brackets highlighter. It's the general parsing issue. And there's actually two of them 😄
I will open separate issues...

[Iskustvo]

Nah, it's unrelated to brackets highlighter. It's the general parsing issue. And there's actually two of them 😄
I will open separate issues...

Here's the first one: #891

For the second one, it's related to some default zsh settings that I turned off in my configuration, so it's not reproducible in clean environment. If I ever get too annoyed by it, I'll find exactly which setting is triggering it and open the issue then.
For the reference (if someone else runs into this) it looks like this:

It can be reproduced in clean environment just by sourcing this version of my config file and z-sy-h with only main highlighter enabled. Enabling the brackets highlighter as well will hide the issue in the current version, since it would override highlighting for the closing bracket. As additional info, the issue isn't with any setting that I used, but in the leading unsetopt -m "*" line which reset all zsh options before I enabled the ones that I prefer.

[danielshahaf]

For the second one, it's related to some default zsh settings that I turned off in my configuration, so it's not reproducible in clean environment. If I ever get too annoyed by it, I'll find exactly which setting is triggering it and open the issue then. For the reference (if someone else runs into this) it looks like this:

If you press <Enter> on the line in the screenshot, does it (1) define a function named aaa whose body is echo "aaa", or (2) print $PS2? In the former case, the red highlight is a bug; in the latter, z-sy-h is functioning as intended (up to #695).

It can be reproduced in clean environment just by sourcing this version of my config file and z-sy-h with only main highlighter enabled. Enabling the brackets highlighter as well will hide the issue in the current version, since it would override highlighting for the closing bracket. As additional info, the issue isn't with any setting that I used, but in the leading unsetopt -m "*" line which reset all zsh options before I enabled the ones that I prefer.

That unsetopt -m "*" line shouldn't make much of a difference since few options are set by default (run setopt without arguments in zsh -f to see that). What should make a difference is the IGNORE_CLOSE_BRACES option. Setting that option causes behaviour (2).

[Iskustvo]

If you press <Enter> on the line in the screenshot, does it (1) define a function named aaa whose body is echo "aaa", or (2) print $PS2? In the former case, the red highlight is a bug; in the latter, z-sy-h is functioning as intended (up to #695).

This use-case is certainly (1).

That unsetopt -m "*" line shouldn't make much of a difference since few options are set by default (run setopt without arguments in zsh -f to see that).

Unfortunately, this isn't really as simple...
setopt shows just a few options which differ from the default configuration:

noglobalrcs
interactive
monitor
norcs
shinstdin
zle

However, running it after setopt -m "*", it shows quite a lot of no[option]s:

noaliases
noalwayslastprompt
noappendhistory
noautolist
noautomenu
noautoparamkeys
noautoparamslash
noautoremoveslash
nobadpattern
nobanghist
nobareglobqual
nobeep
nobgnice
nocaseglob
nocasematch
nocheckjobs
nocheckrunningjobs
noclobber
nodebugbeforecmd
noequals
noevallineno
noflowcontrol
nofunctionargzero
noglob
noglobalexport
noglobalrcs
nohashcmds
nohashdirs
nohashlistall
nohistbeep
nohistsavebycopy
nohup
interactive
nolistambiguous
nolistbeep
nolisttypes
nomultibyte
nomultifuncdef
nomultios
nonomatch
nonotify
nopromptcr
nopromptpercent
nopromptsp
norcs
shinstdin
noshortloops
nounset

For example, without ZLE and GLOB options, this plugin can't be successfully sourced.
Same happens if IGNORE_CLOSE_BRACES is enabled before sourcing of z-sy-h.
The minimal example for reproduction of this I currently have is:

unsetopt -m "*"
setopt ZLE GLOB
source "/usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh"
ZSH_HIGHLIGHT_HIGHLIGHTERS=(main)

[Iskustvo]

We have diverged a bit.
Do you have comment about discussion for main issue?