Scandinavian keyboard layout can't handle input tilde (~)

[ixrock]

To type ~ with Finnish/Swedish keyboard I have to press hotkey alt+^:

After that it shows something similar to ~ but right after when I press any other character (except Enter) it will erase "pre-tilde" symbol. Press Enter "converts" to normal ~ and nothing more (without real Enter).

I checked how this works with vanilla JS and it seems something wrong at browser-level:

Is it fixable? Wdyt?

Related bugs:

#174

Details

• Browser and browser version: Chrome (latest)
• OS version: MacOS X (latest)
• xterm.js version: 3.14.0

Steps to reproduce

1. Add support of Finnish keyboard in your OS
2. Try to use ~ from keyboard

[Tyriar]

VS Code issue: microsoft/vscode#68262

[ixrock]

@Tyriar how xterm related to vscode?

[Tyriar]

vscode uses xterm, that's the issue that's been reported over there.

[gdlx]

Terminus has the same issue: Eugeny/tabby#768
This isn't limited to Scandinavian keyboards. At least French and Spanish are affected too (any layout with the AltGr key and 2 steps accents feature I presume).

[ixrock]

It's still possible to catch ~ with InputEvent:

[Tyriar]

I'm not sure if this ever worked if InputEvent fixes the problem, sounds like it might be the same root cause as #469 and needs us to move input handling over to input instead of the key events.

[ixrock]

@Tyriar Using only InputEvent for sure not enough. Its evt.data just contains current value-string for input/textarea and nothing about pressed keys, etc. I just thought it might help to fix this issue combined with normal KeyboardEvent.

[goulartdev]

I'm having the same issue in Fluent Terminal.
I can't type ~, it shows Þ instead. My keyboard is PT-BR (with the AltGr key), but if I change the layout to US, I can type ~ using the ' key.

[insertt]

Until this issue will be fixed I suggest using workaround from here

[GitSquared]

I'm not sure if this exactly the same issue, but I find that typing any diacritics (like ö ầ etc), not just the tilde, on xterm doesn't work at all (prints like o a, without any diacritic applied). Tested with french keyboard and Linux, latest xTerm.js on Hyper and/or eDEX-UI.

[Tyriar]

@GitSquared I think this might be related to your $LANG variable?

[GitSquared]

@Tyriar My $LANG is set to en_US.UTF-8 - but typing diacritics does work in other terminal emulators (Konsole), and switching to, for instance, fr_FR.UTF-8 does not resolve the issue.

Also, I'm unsure how $LANG affects typing input? Is this some xterm.js-specific behavior?

[jerch]

@Tyriar Is this only an issue under Mac or with Mac keyboards (MBP)? I have no problems with PC keyboards with german keyboard layout - all third level shifts work as intented in all browsers (tested with demo in chrome + FF).

Some ideas whats going wrong (in fact it looks like 2 separate issues on the lower level):

• original poster reported to press Alt + ^: Alt has a special meaning by default, we map it to the Meta key for PC keyboards (as it lacks a dedicated Meta key). This assumption might be wrong for some keyboard types (Mac?). For Mac keyboards we have an macOptionIsMeta setting, we might have to check if this needs a slightly different Alt+XY key handling in Keyboard.ts as well if set.
• fluentTerminal and AltGr issue: This looks like the browser engine reports the wrong key/keyCode (not applying keyboard layout correctly?). When AltGr is held down normally the reported entry should shift to the third character for the second key, if browser do wrong here I fear there is not much we can do in the first place. But atm this is all speculation and needs proper bug testing to catch all edge cases. If InputEvent still reports the right key value we might get away with a combined KeyboardEvent + InputEvent processing. If this does not work - a last resort might be to deliver our own (customizable) keyboard layout maps.

Key handling always has been a pain in the a** in browsers, maybe we can learn something from the monaco guys here?

Things mentioned in this issue are hard to fix as long as we cannot repro it and dont know the exact circumstances this bug shows up (OS + browser version). Any volunteers to (partly) address this?

To test if the browser reports the right thing you can use https://w3c.github.io/uievents/tools/key-event-viewer.html.

[Tyriar]

Is this only an issue under Mac or with Mac keyboards (MBP)?

@jerch you might be right OP and the 2 linked vscode issues are all on macos.

[laarmen]

FYI, a string of closed/duplicate issues on the VSCode repository lead me to this issue, and I'm on Windows. Using a French AZERTY layout, the tilde is on the "number" row, on the "2" key, which has as main char "é", shift char "2" and third char "~", which normally has a dead key behaviour. Thus, to type in a ~ character, the normal sequence would be AltGr+é, Space. However, when doing so the result is é~.

I'm not entirely certain it is the exact same bug, but I'm assuming it stems from the same root causes.

[jerch]

@laarmen It seems we first have to identify the problematic layouts, AZERTY seems to be one of them. Our problem is that none of us can dig into that, we only use english/german keyboard layouts.

So if you dont mind - could you also check if the other third shifts are wrong as well (I assume so)? And whether https://w3c.github.io/uievents/tools/key-event-viewer.html reports the right thing - if so its prolly a problem on our side.
Please also report us your OS version and installed language / keyboard layout. (Not sure yet, I think the browser tries to apply the OS keyboard layout setting for the event codes, thus this might also be a source of trouble depending on the configuration).

Edit: Here is a list of common latin keyboard layouts https://en.wikipedia.org/wiki/List_of_Latin-script_keyboard_layouts (and links to other layouts in the footer). And the ISO spec for keyboards https://en.wikipedia.org/wiki/ISO/IEC_9995.

[laarmen]

OK, so I tried to reproduce on a Docker xterm.js instance, and couldn't reproduce (Firefox and Chrome both). I'm on Windows 10, and my bug happens in the VSCode terminal (1.38.1). 🤷‍♂

[jerch]

@laarmen Oh thats weird, Im pretty sure vscode runs exactly the same code for key input unaltered.

@Tyriar FYI. Maybe an electron bug not correctly applying OS keyboard layout?

[Tyriar]

@rebornix have you seen any reports around this for monaco?

[ssh-peppe]

For me on macOS, Finnish keyboard, with Safari and Chrome a tilde + space results in ~ and tilde + n results in ñ as it should. But on Firefox a tilde + space results in ~~ and tilde + n results in ~ññ.

Interestingly this behavior difference can be seen (but a bit differently for some reason) on the https://xtermjs.org/ home page example terminal. If you enter tilde + space there on Safari you get nothing at all. Same for tilde + n. But with Firefox while tilde + space gets you nothing a tilde + n gets you ñ. Not sure why it's working differently there.

Debug logging shows the following when typing tilde + space on Firefox:

xterm.js: sending data "~" Array [ 126 ]
xterm.js: sending data "~" Array [ 126 ]
xterm.js: parsing data ~~

For tilde + n it shows:

xterm.js: sending data "~" Array [ 126 ]
xterm.js: sending data "ñ" Array [ 241 ]
xterm.js: sending data "ñ" Array [ 241 ]
xterm.js: parsing data ~
xterm.js: parsing data ññ

Note that there are differences in browsers with regards to what is reported from e.g. compositionend vs. input vs. keydown vs. keypress events.

I should add that nothing is being sent until you hit space or n. That is, the initial tilde that starts the character compose does not result in any callbacks firing, whch is correct.

[alexandrehpiva]

As @Tyriar set that issue as a duplicate of this one, I am posting here that nothing solves for me.

I tried to re-install VSCode, change terminal to use ZSH shell, change the VSCode language and added configuration "terminal.integrated.env.linux" to use UTF-8.

I personally don't think this is the same problem of that issue because I am using Ubuntu and when I type (in VSCode terminal) a special char alone and press space it works ex.: "^~`'", but when I type a special char like "^" + "a" only show "a" instead of "â". Same with "áâãéẽíĩóú". But in normal Ubuntu terminal everything works ok.

Does anyone have any more ideas?

[jerch]

@alexandrehpiva Can repro that under ubuntu with chrome and QWERTZ (german layout), where ^ is a dead key. I think dead key composition never worked correctly for chrome under linux, while it works with firefox.

Can you confirm - is ^ a dead key in your keyboard layout? (dead key - not producing a symbol on first press, instead adding some modification to a following key, often used to add ^, ~, ¨ or other accents to a letter)

[Eugeny]

@Tyriar bad news, found out that some Windows machines just don't produce the "Dead" keydown no matter what. I'm investigating and I think we should reopen this for the time being.

[Eugeny]

Looks like it's producing a bogus version of a third level shift with AltGraph instead of alt and missing modifier flags - here's the broken version:

and a normal third level shift:

[Tyriar]

😦

[fperez]

I am seeing the same problem (which also impacts JupyterLab, where we use xterm.js) and wanted to mention that there's an easy way even for developers on US keyboards to reproduce it, which may help track it down: I'm on a Mac, and if you install the "US International" layout:

that will give you dead keys for accents in western languages (I use it as I have to type regularly in English, Spanish and French + coding and my keyboards are US-layout). With this layout active, to type say cd ~ I will type cd , then tilde, then space. The space keystroke makes the tilde dead-key become the normal tilde (instead of say ~ + n, which will become ñ with that layout.).

As of right now, testing this on the xtermjs.org terminal with this layout does not work (nor do other accents).

I hope this helps the dev team perhaps track it down better - we'd greatly benefit from a fix to this on the JupyterLab team... LMK if I can provide further testing/feedback (no fixing offered - I don't know this codebase/JS well enough to help, sorry!!). Otherwise thanks for the fabulous tool!

[alexandrehpiva]

Can you confirm - is ^ a dead key in your keyboard layout? (dead key - not producing a symbol on first press, instead adding some modification to a following key, often used to add ^, ~, ¨ or other accents to a letter)

No, it isn't. My keyboard is pt-BR Brazilian Portuguese. Really, my problem explanation is like the
issue microsoft/vscode#105051

[fperez]

@alexandrehpiva do you still get the problem on VSCode? I just tested it on my current setup (VSCode 1.59.1 on MacOS Big Sur, with US International layout) and it's now fine. Just checking to see if with current versions of VSCode it still happens, to help narrow down things.

[alexandrehpiva]

@alexandrehpiva do you still get the problem on VSCode? I just tested it on my current setup (VSCode 1.59.1 on MacOS Big Sur, with US International layout) and it's now fine. Just checking to see if with current versions of VSCode it still happens, to help narrow down things.

Yes, I just update to 1.60.0, and still same problem. I'm unable to use accented characters with Ubuntu bash inside VSCode. I tried to install and use Oh My ZSH too and still same. The mad thing is that work's perfectly in terminal outside VSCode. Anyone knows how to change configuration for this (if exists) in Ubuntu 20.04 LTS?

[alexandrehpiva]

I recorded an example:

[alexandrehpiva]

I just discovered a strange thing: If I press the accent key 3 times and then press the letter, it works and I get: "~ã"

[thiagobraga]

I just discovered a strange thing: If I press the accent key 3 times and then press the letter, it works and I get: "~ã"

I realized the same thing today. If I press the tilde key followed by the "a" key, it is displayed a in the terminal and not ã. But if I press the tilde three times and then the "a" key, ~ã is produced. I imagine the issue should not be closed yet. I'm on Ubuntu 20.04.3 and on Terminator and Gnome Terminal the accent works perfectly.

[devphobia]

Ubuntu 20.04 using GNOME and having the same issues.
The integrated terminal doesn't seem to allow me to input characters like [é, ó, ú, ô, etc...] while all my other terminals work just fine. Even installed Atom to see if the bug persisted on other code editors but it seems it's related to VSCode and VSCode only.
Any ideas on how to get rid of this?

[mauriciosantosds]

Same problem here with vscode integrated terminal in Ubuntu 20.04 GNOME.

[renathoaz]

For those using Ubuntu 20.04.
After a long searching I found out that vscode depends on ibus for the accents work on integrated terminal.
the workaround :

1.Install ibus: sudo apt install ibus ibus-gtk3
2. Create the config file:

cat <<EOF >ibus.dconf
[general]
preload-engines=['xkb:br::por']

[general/hotkey]
triggers=@as []

[panel]
show-icon-on-systray=false
EOF

3. import it to dconf: dconf load /desktop/ibus/<ibus.dconf
4. init ibus with proper config: im-config -n ibus
5. activate ibus on GNOME: gsettings set org.gnome.desktop.input-sources sources "[('ibus', 'xkb:br::por')]"

and now everything is just working fine:

[thiagobraga]

Now I'm not using Ubuntu 20.04 anymore, I upgraded to Ubuntu 21.10 and VSCode 1.66. I noticed that ibus and ibus-gtk3 are already installed. I believe that since I reinstalled, I hadn't done the accent test in VSCode terminal. And to my surprise, it now worked normally, without having to change any settings.