The VI-mode commands f' , f" , t' and t" insert ' or " instead of moving the cursor when running on Windows when using the International Layout (dead keys)

[thomazmoura]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues, especially the pinned issues.

Exception report

N/A

Screenshot

Print showing that when I try to use F" or F' it prints the character instead of moving the cursor to the previous ocurrence of the character.

Environment data

PS Version: 7.2.13
PS HostName: ConsoleHost
PSReadLine Version: 2.2.6
PSReadLine EditMode: Vi
OS: 10.0.19041.320 (WinBuild.160101.0800)
BufferWidth: 120
BufferHeight: 9001

Steps to reproduce

(The following steps can be made replacing ' for " or `)

• Set the Windows language layout to United States-International
• Write anything with an ' on it on PowerShell
• Press ESC to go the VI-mode's normal mode
• Try to move to the written character with f' or t' (or if the character is to the left of the cursor with F' or T')

(the key here is used to send the ' key directly instead of merging it with another key since in this layout it's a dead key)

Expected behavior

The cursor should move to the desired character, exactly like it happens on the US layout (the only difference should be the need to press space after ' or " or `).

Actual behavior

When running it on WSL2 it works exactly as expected - since it's a dead key on US International Layout when I press f " it waits the next key and if I press space (to insert the " character) it searches for next ocorrence of the " character. But when running on Windows it doesn't seem to recognize the dead key and fails the command as soon as I press any of it.

I've noticed it sends a bell as soon as I press the dead key (if I press d f and then ' as soon as the ' is first pressed it sounds the bell), which I believe indicates that the previous command failed/was cancelled and then when I press space it prints the character (if I press the same accent again it prints it twice) which would be the expected behaviour if I was on insert mode and not chaining commands on normal mode.

I've made tests with the US keyboard layout and it works fine. Also, some accented words like à or á (that are written with the keys a backtick and a ' trigger the same behavior, just printing the character instead of moving the cursor.

As an additional info the ^ key is also a dead key on this layout but has a different behaviour - it doesn't ring any bell, and when chained to command it just ignores the command and goes to the start of the line (the expected behaviour when the command is like d^ would for it to delete until the start of the line, not just go there).

I've also made many tests like using Windows Terminal or conhost and using both PowerShell 5 and PowerShell 7 versions (with the latest preview of PSReadLine). So far it only happens on Windows and when using the United States-Internatial layout (which is the one I use to be able to write both english and portuguese on my ANSI layout keyboard)

[thomazmoura]

Just tested on VS Code and it works there, using the same PowerShell and PSReadLine versions.

Also, I've realized I wasn't using the latest pre-release version of PSReadLine and I've just updated it - but on Windows Terminal and conhost the problem persists even on 2.3.2 (beta2)

[thomazmoura]

Here's a exception report from a related issue that happens from time to time but I still haven't managed to discover exactly when (it always happens after trying to go to a ' or a ", realizing it won't work on Windows and trying to workaround moving to another character instead (usually a -).

The command I was using is Criar-PullRequest.ps1 -Titulo 'Implementação de criação de campanhas e formulário de pesquisa' -Equipe Arquitetura -AbrirPullRequest (it's a custom script I have) - I was trying to change all the text from the first ' to the second ' so I could rerun the command with a different text. When it failed, I tried changing to the - character and then the exception was thrown.

---

Oops, something went wrong.
Please report this bug with ALL the details below, including both the 'Environment' and 'Exception' sections.
Please report on GitHub: https://github.com/PowerShell/PSReadLine/issues/new?template=Bug_Report.yaml
Thank you!

Environment

PSReadLine: 2.3.2-beta2
PowerShell: 7.2.13
OS: Microsoft Windows 10.0.19045
BufferWidth: 144
BufferHeight: 40

Last 200 Keys:

p s 1 Spacebar - T i t u l o Spacebar ' I m p l e m e n t a ç ã o Spacebar d e Spacebar c r i a ç ã o Spacebar d e Spacebar c a m p a n h a s Spacebar e Spacebar f o r m u l á r i o Spacebar d e Spacebar p e s q u i s a ' Spacebar - E q u i p e Spacebar A r q u i t e t u r a Spacebar - A b r i r P u l l R e q u e s t Escape I # Enter # End Escape Shift+ ^ x Enter Escape k f ' h x f - ; ; ; , , l l l l l l l l l l l h h c t ' h x f - F ' h x h h h h h h h h h h h h h h h h h h h h h h h h h B B B B B B l c t -

Exception

System.InvalidOperationException: Operation is not valid due to the current state of the object.
at Microsoft.PowerShell.PSConsoleReadLine.StartEditGroup()
at Microsoft.PowerShell.PSConsoleReadLine.GroupUndoHelper.StartGroup(Action2 instigator, Object instigatorArg) at Microsoft.PowerShell.PSConsoleReadLine.ViReplaceToBeforeChar(Char keyChar, Nullable1 key, Object arg)
at Microsoft.PowerShell.PSConsoleReadLine.ViReplaceToBeforeChar(Nullable1 key, Object arg) at Microsoft.PowerShell.PSConsoleReadLine.ProcessOneKey(PSKeyInfo key, Dictionary2 dispatchTable, Boolean ignoreIfNoAction, Object arg)
at Microsoft.PowerShell.PSConsoleReadLine.ViChordHandler(Dictionary2 secondKeyDispatchTable, Object arg) at Microsoft.PowerShell.PSConsoleReadLine.ViChord(Nullable1 key, Object arg)
at Microsoft.PowerShell.PSConsoleReadLine.ProcessOneKey(PSKeyInfo key, Dictionary2 dispatchTable, Boolean ignoreIfNoAction, Object arg) at Microsoft.PowerShell.PSConsoleReadLine.InputLoop() at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics, CancellationToken cancellationToken, Nullable1 lastRunStatus)

[StevenBucher98]

Interesting bug, able to reproduce and also seems to work with Mac OS Terminal. @DHowett any ideas if this is an issue with Windows Terminal/conhost? (using latest 2.3.3 version of PSRL)

[thomazmoura]

I've just tested on 2.3.4 (though the comment asking for this seem to have disappeared) and the behavior seem to be the same still. It's like when I press a dead key it "counts" already as part of the combo, but it's not recognized so it breaks the chain. When I press space (or any other key for that matter) to actually send the proper character the chain is already broken so it inserts the character normally like there was no command at all.

I believe the way to fix it is probably ignoring the dead key presses for the command chaining (which seems to be what happens on Linux, for example), but I have no idea how to do that or where the code for this command chaining is.

I might be able to take a look if there is any documentation or instructions on where in the code that part is processed.

[thomazmoura]

@StevenBucher98, could you help me run the PSReadLine locally or point to some documentation or tutorial for it? I've tried building the PSReadLine project, starting a new PowerShell (pwsh) session on the bin/Debug/net6.0 folder, Removing the pre-loaded PSReadLine and then importing the PSReadLine.psm1 file, but it doesn't seem to load any the changes I made (currently I'm just throwing some exceptions to garantee that I'm running the changed code and not the default one). I've also tried running Import-Module .\Microsoft.PowerShell.PSReadLine2.dll as suggested by copilot, but it gives the error Import-Module: Assembly with same name is already loaded. I found no way to start a new pwsh session without PSReadLine pre-imported so I could import the local version.

Just to give an update I've so far found that the code which is likely the culprit is the SearchChar method on ReadLine.vi.cs (line 166). There it calls ReadKey() and after getting the value proceeds to make the movement. The part where key is actually read is not shown on the ReadKey() method (it seems to happen on another thread) but my guess is that it runs something similar to calling [Console]::ReadKey() on pwsh and that's where Windows and Linux diverge considerably. On Linux, any dead keys are ignored by [Console]::ReadKey() - only when the next character is typed it returns the combined character (like á, or ã) that way on layouts that have ', " or ` as dead keys it waits for a another key (like space) to return the key. On windows though, this same method returns immediately when a dead key is typed, but with a null/empty KeyChar. The PSReadLine reads this empty KeyChar and then does nothing. When the user finally types the inteded key the chain has already been broken by the dead key.

Since resolving this on the dotnet side seems way more troublesome (since any change there can impact the whole ecosystem and there many low level considerations on these changes) and this bug is currently affecting only the VI mode on PSReadLine for layouts with dead keys, what I would like to attempt is to have a loop on the ReadKey() (or wrapped around on the ReadKey in places like the SearchChar method and similar methods) so that when a user presses a dead key PSReadLine continues waiting or ignore it and then call ReadKey again, until a valid key is pressed. But to do this I need a way to run PSReadLine locally.

[thomazmoura]

@StevenBucher98 I've managed to be able to test locally (basically all I needed was pwsh -NoProfile -NonInteractive).
Then I found the issue made a PR (#3928) could you help me getting someone to review it?