Any Wayland support plans?

[crispinb]

I see in the documentation that Wayland isn't currently supported. Are there any plans (however tentative) to do so at some point? I ask because I'd like to know whether it's worth my while following espanso's releases/progress.

[federico-terzi]

Hey @crispinb ,

Thanks for your interest in espanso! I definitely don't exclude the possibility of supporting Wayland in the future, but it will surely take a while (many months) as it would require a non-trivial rewrite of the espanso core.
Your best bet is to keep the notifications on this issue so that you get notified of all the updates.

For everybody else: if you are interested in this feature, leave a "like" reaction to the post of @crispinb so that I can have an idea of the demand for this feature.

Cheers :)

[federico-terzi]

As a reference for the future, this project seems to use the APIs we need: https://github.com/ReimuNotMoe/ydotool

Also for the clipboard APIs: https://github.com/bugaevc/wl-clipboard
and even better, a Rust crate: https://github.com/YaLTeR/wl-clipboard-rs

Got a basic POC of the key detection working using the "input.rs" library
inputtest.tar.gz

Which should be coupled with logkeys mappings to obtain the characters back: https://github.com/kernc/logkeys/tree/master/keymaps

[MohammedAl-Mahdawi]

Hi @federico-terzi,
Any estimated date when this feature become available?

[federico-terzi]

Hey @MohammedAl-Mahdawi,

Thank you for the interest! It's hard to estimate the date, as for the upcoming 3/4 months I'll be very busy with university, so it's unlikely I'll be able to tackle this feature just yet (as it requires a non-trivial rewrite of the core).
What I can say is that it's on top of my priority list, as this one is by far the most requested feature.
Two things to keep in mind:

1. I've done a bit of preliminary testing, and I'm reasonably sure we can port espanso to Wayland. What I'm not sure of is whether we will be able to port all the features, as the techniques we will use are pretty different from the current X11 implementation.
2. I still have to figure out the optimal way to handle the permissions of this feature, as we need to go deeper in the stack by exploiting /dev/input. That's not possible for regular user-level programs, so we either need to:
   • execute espanso as root. If we choose this route, we will need to figure out the best way to de-elevate the privileges when executing custom scripts or shell commands, as otherwise, we would be running them as root which is super dangerous.
   • add the user to a group which has read/write permissions to /dev/input, but of course that would mean giving also the other user-level processes the same permissions, which is not ideal.
   • the third possible way (but I still haven't got the time to investigate it properly), would be to create another espanso user and add it to the group which can read/write to /dev/input, and execute the espanso daemon as that user. This seems like the best approach to me (as only espanso would benefit from these permissions), but I don't know yet what consequences we could face,

I'm really open to ideas and proposals, as your experience really matters here! So if you have an idea, don't hesitate to write them here.

Cheers :)

[dzintars]

Also would love to see this happen. Switched to the Wayland/Sway a while ago and... somewhat missing Espanso there. 😃
[ERROR] Error, can't connect to X11 context 😞
BTW: On espanso start there could be at least feature detection with some human friendly feedback like "Wayland is not supported" or smt.

[federico-terzi]

I've got some great news! As part of the major refactor happening in the dev1.x branch (in which I'm rewriting espanso from scratch), I was finally able to implement the backends needed to support Wayland. In other words, Wayland support is coming!

It will take a while as there are still a lot of modules to rewrite, but the hardest part is now done!
I'll make sure to create an issue to track the refactor soon.

wayland.mp4

[gusbemacbe]

@federico-terzi

I used espanso-git that triggers the branch dev. I tested, everything worked, except the native terminals, including GNOME Terminal, which is X11. But it works on the integrated terminal of VSCode.

[federico-terzi]

@gusbemacbe That's expected, given that the dev branch still contains the old (0.7.4) version and there is no code to support wayland there :)
Development of the new version is happening on the dev1.x branch, which includes support for wayland but is still in progress (I'd say 60% complete). I'll let you know when an alpha version is ready to be tried

[weitzman]

I came looking for this. Hoping this can be worked on soon. Cheers.

[reesericci]

@federico-terzi any updates?

[federico-terzi]

@reesericci Thank you for the interest!

any updates?

Yes, the upcoming version 2 will ship with experimental Wayland support, hopefully by the end of September. Right now, there are two major things I still need to figure out:

• The optimal way to detect the currently active application. Without this, espanso can't support app-specific configuration on Wayland. There seems to be some Gnome-specific and Sway-specific ways to accomplish that, but I still haven't properly investigated.
• The best packaging strategy. For the alpha, espanso will probably need to be compiled manually on Wayland (it's really not too difficult, just a couple of terminal commands :) )

Keep in mind that the Wayland ecosystem is so fragmented that it's hard for me to make sure everything works as expected on all distros. So far, I've been testing it in a Fedora VM (with Gnome), but in theory, it should work on Sway as well.

Hopefully, with some help from the community, we'll be able to get something stable by the end of the year :)

[neilzone]

Keep in mind that the Wayland ecosystem is so fragmented that it's hard for me to make sure everything works as expected on all distros.

I can test it on Debian 11 (Bullseye).

[Sonique]

Definitely will test on ubuntu/sway  😀

[federico-terzi]

@neilzone @Sonique Thank you guys! Your help will be invaluable :) I'll make sure to write an announcement with the basic steps to get started soon

[mistical2008]

@reesericci Thank you for the interest!

any updates?

Yes, the upcoming version 2 will ship with experimental Wayland support, hopefully by the end of September. Right now, there are two major things I still need to figure out:

* The optimal way to detect the currently active application. Without this, espanso can't support app-specific configuration on Wayland. There seems to be some Gnome-specific and Sway-specific ways to accomplish that, but I still haven't properly investigated.

* The best packaging strategy. For the alpha, espanso will probably need to be compiled manually on Wayland (it's really not too difficult, just a couple of terminal commands :) )

Keep in mind that the Wayland ecosystem is so fragmented that it's hard for me to make sure everything works as expected on all distros. So far, I've been testing it in a Fedora VM (with Gnome), but in theory, it should work on Sway as well.

Hopefully, with some help from the community, we'll be able to get something stable by the end of the year :)

I can test it on Manjaro (Archlinux derivative) with Gnome DE and Sway.

[gusbemacbe]

I advice espanso's wxgtk2 can cause a continuously bothersome bug on Wayland, making Wayland icon appearing and disappearing several times on the menubar, and makes the screen blinking.

[federico-terzi]

@gusbemacbe Thank you for the feedback! Not sure if that's due to wxgtk2 or the way Espanso handles the clipboard on Wayland. From the docs, there's a limitation that might be related:

If you are using Gnome, Espanso causes a small "flicker" when using the clipboard backend. This does not happen on Sway.

[gusbemacbe]

@gusbemacbe Thank you for the feedback! Not sure if that's due to wxgtk2 or the way Espanso handles the clipboard on Wayland. From the docs, there's a limitation that might be related:

If you are using Gnome, Espanso causes a small "flicker" when using the clipboard backend. This does not happen on Sway.

I'm going to test on GNOME.

Observe that espanso's wgtk3 is also affected.

If you want to investigate profoundly, install Endeavour OS with KDE and Wayland on a virtual machine.

[Delanii]

I don't know if this is the correct place to ask, but I have just installed espanso on my Manjaro KDE.
I have tried both alpha and stable versions and both on X and Wayland.
I was very surprised that on Wayland the alpha version wasn't working (not sure about the specific issue, but I guess that the service/deamon wasn't starting).
But when I installed the stable version on X, after logging into Wayland it is still working ...
How is that possible?? :D

[gusbemacbe]

I don't know if this is the correct place to ask, but I have just installed espanso on my Manjaro KDE.
I have tried both alpha and stable versions and both on X and Wayland.
I was very surprised that on Wayland the alpha version wasn't working (not sure about the specific issue, but I guess that the service/deamon wasn't starting).
But when I installed the stable version on X, after logging into Wayland it is still working ...
How is that possible?? :D

@Delanii, it does work, but you will be able to trigger the keys on only the apps built in X. You'll not be able to trigger the keys in Konsole.

[reesericci]

@federico-terzi Is there any chance for a flatpak? That's the native format for Fedora Silverblue, as well as other immutable distros. e.g. Steam Deck. Flatpak is also winning the universal packaging wars across all distros.

[jokeyrhyme]

@reesericci there's already a flatpak discussion over here: #386

[reesericci]

perfect

[hseliger]

Detecting applications

This is hard. Wayland is supposed to be secretive about this. No application should know about any other application. AFAIK current methods are not really intended for what espanso wants to achieve. On Sway for example one can use IPC to get the window tree (and also which window is focused). A better approach would probably be some Wayland protocol, which can be used to at least identify applications.

Hi!
Just stumbled over this discussion. I recently had a similar issue and could at least get the window class identified under Gnome/Wayland. It does require an extension, though ("Window Calls", https://github.com/ickyicky/window-calls). Then a gdbus call yields the active window class:

gdbus call --session --dest org.gnome.Shell --object-path /org/gnome/Shell/Extensions/Windows --method org.gnome.Shell.Extensions.Windows.List \
  | sed -E -e "s/^\(.\[//g" -e "s/\},\{/\}\n\{/g" \
  | grep '"focus":true' \
  | sed -E -e 's/^.*class":"//' -e 's/".*$//'

Maybe this helps.

[federico-terzi]

@hseliger Thanks for sharing!

[hseliger]

Sure. The same call, btw, also returns the details on PID and window title. the gdbus cal returns a JSON, the active window has the entry "focus" set to true. You get e.g.
{"class":"gnome-terminal-server","pid":185648,"id":3721670068,"maximized":0,"focus":true,"title":"user@host: ~/src/ActiveWindowInfo/active-window"}, so you can extract all the info you need.

I briefly tried to dig into the gtk-rs documentation, but I don't have the time to figure out how to do the gdbus call from rust to build an example.

[reesericci]

When will there be a Wayland Flatpak or AppImage?

[Schlaefer]

It stopped working for me on KDE plasma 5.25. Insta-Crash on startup.

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007f86c58293d3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2  0x00007f86c57d9838 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007f86c57c3535 in __GI_abort () at abort.c:79
#4  0x00005589396cf0d3 in wayland_client::imp::proxy::proxy_dispatcher<wayland_client::protocol::wl_shm::WlShm>
(_implem=0x55893a65ac73 <wayland_sys::RUST_MANAGED>, proxy=0x7f86ac03ce90, opcode=0, _msg=0x55893ae0df88 <wayland_client::protocol::wl_shm::wl_shm_events>, args=0x7f86ac02cf28)
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.28.6/src/native_lib/proxy.rs:459
#5  0x00007f86c6beacd8 in wl_closure_dispatch (opcode=0, target=0x7f86ac03ce90, dispatcher=<optimized out>, closure=0x7f86ac02cf10)
at ../wayland-1.20.0/src/connection.c:1034
#6  dispatch_event (display=display@entry=0x7f86ac03d600, queue=0x7f86ac004160, queue=<optimized out>)
at ../wayland-1.20.0/src/wayland-client.c:1577
#7  0x00007f86c6beaf8c in dispatch_queue (queue=0x7f86ac004160, display=0x7f86ac03d600)
at ../wayland-1.20.0/src/wayland-client.c:1729
#8  wl_display_dispatch_queue_pending (display=display@entry=0x7f86ac03d600, queue=queue@entry=0x7f86ac004160)
at ../wayland-1.20.0/src/wayland-client.c:1971
#9  0x00007f86c6bec943 in wl_display_dispatch_queue (queue=<optimized out>, display=<optimized out>)
at ../wayland-1.20.0/src/wayland-client.c:1947
#10 0x00007f86c6bed930 in wl_display_roundtrip_queue (display=0x7f86ac03d600, queue=0x7f86ac004160)
at ../wayland-1.20.0/src/wayland-client.c:1358
#11 0x00005589396070ff in wayland_client::imp::event_queue::{impl#0}::sync_roundtrip::{closure#0}<smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>> ()
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.28.6/src/native_lib/event_queue.rs:85
#12 0x0000558939607867 in wayland_client::imp::event_queue::with_dispatch_meta::{closure#0}<core::result::Result<u32, std::io::error::Error>, smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>, wayland_client::imp::event_queue::{impl#0}::sync_roundtrip::{closure_env#0}<smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>>> ()
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.28.6/src/native_lib/event_queue.rs:24
#13 0x00005589395e09f5 in scoped_tls::ScopedKey<core::cell::RefCell<(&mut dyn core::ops::function::FnMut<(wayland_client::RawEvent, wayland_client::proxy::Main<wayland_client::anonymous_object::AnonymousObject>, wayland_commons::filter::DispatchData), Output=()>, wayland_commons::filter::DispatchData)>>::set<core::cell::RefCell<(&mut dyn core::ops::function::FnMut<(wayland_client::RawEvent, wayland_client::proxy::Main<wayland_client::anonymous_object::AnonymousObject>, wayland_commons::filter::DispatchData), Output=()>, wayland_commons::filter::DispatchData)>, wayland_client::imp::event_queue::with_dispatch_meta::{closure_env#0}<core::result::Result<u32, std::io::error::Error>, smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>, wayland_client::imp::event_queue::{impl#0}::sync_roundtrip::{closure_env#0}<smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>>>, core::result::Result<u32, std::io::error::Error>> (self=0x55893ac84d68 <wayland_client::imp::event_queue::DISPATCH_METADATA>, t=0x7f86c381fa58, f=...)
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137
#14 0x00005589396074e7 in wayland_client::imp::event_queue::with_dispatch_meta<core::result::Result<u32, std::io::error::Error>, smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>, wayland_client::imp::event_queue::{impl#0}::sync_roundtrip::{closure_env#0}<smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>>> (fb=..., data=..., f=...)
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.28.6/src/native_lib/event_queue.rs:24
#15 0x000055893960705a in wayland_client::imp::event_queue::EventQueueInner::sync_roundtrip<smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>>
(self=0x7f86ac00d9e0, data=..., fallback=...)
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.28.6/src/native_lib/event_queue.rs:83
#16 0x00005589395fd585 in wayland_client::event_queue::EventQueue::sync_roundtrip<(), smithay_client_toolkit::environment::{impl#0}::new::{closure_env#1}<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>>
(self=0x7f86c381ffa0, data=0x7f86c381fea8, fallback=...)
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.28.6/src/event_queue.rs:203
#17 0x00005589395d463b in smithay_client_toolkit::environment::Environment<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>::new<espanso_detect::evdev::sync::wayland::EspansoModifiersSync> (display=0x7f86c38200a8, queue=0x7f86c381ffa0, env=...)
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/smithay-client-toolkit-0.14.0/src/environment.rs:123
#18 0x000055893958a330 in espanso_detect::evdev::sync::wayland::get_modifiers_state::{closure#4} (display=...)
at /home/siezi/.cargo/registry/src/github.com-1ecc6299db9ec823/smithay-client-toolkit-0.14.0/src/lib.rs:379
#19 0x00005589395c27ff in core::result::Result<wayland_client::display::Display, wayland_client::display::ConnectError>::and_then<wayland_client::display::Display, wayland_client::display::ConnectError, (smithay_client_toolkit::environment::Environment<espanso_detect::evdev::sync::wayland::EspansoModifiersSync>, wayland_client::display::Display, wayland_client::event_queue::EventQueue), espanso_detect::evdev::sync::wayland::get_modifiers_state::{closure_env#4}> (self=..., op=...)
at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/result.rs:1332
#20 0x00005589395eba2b in espanso_detect::evdev::sync::wayland::get_modifiers_state ()
at espanso-detect/src/evdev/sync/wayland.rs:19
#21 0x00005589395d5b2d in espanso_detect::evdev::{impl#1}::initialize (self=0x7f86ac001360) at espanso-detect/src/evdev/mod.rs:136
#22 0x0000558939329013 in espanso::cli::worker::engine::funnel::init_and_spawn::{closure#0} ()
at espanso/src/cli/worker/engine/funnel/mod.rs:71
--Type <RET> for more, q to quit, c to continue without paging--
#23 0x00005589392d074f in std::sys_common::backtrace::__rust_begin_short_backtrace<espanso::cli::worker::engine::funnel::init_and_spawn::{closure_env#0}, ()> (f=...) at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/sys_common/backtrace.rs:122
#24 0x00005589392104b3 in std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure#0}<espanso::cli::worker::engine::funnel::init_and_spawn::{closure_env#0}, ()> () at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/thread/mod.rs:498
#25 0x0000558939289733 in core::panic::unwind_safe::{impl#23}::call_once<(), std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<espanso::cli::worker::engine::funnel::init_and_spawn::{closure_env#0}, ()>> (self=..., _args=())
at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/panic/unwind_safe.rs:271
#26 0x00005589392d4250 in std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<espanso::cli::worker::engine::funnel::init_and_spawn::{closure_env#0}, ()>>, ()>
(data=0x7f86c3822268) at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/panicking.rs:492
#27 0x00005589392d48bb in __rust_try ()
#28 0x00005589392d3da7 in std::panicking::try<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<espanso::cli::worker::engine::funnel::init_and_spawn::{closure_env#0}, ()>>> (f=...)
at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/panicking.rs:456
#29 0x00005589392be1a3 in std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<espanso::cli::worker::engine::funnel::init_and_spawn::{closure_env#0}, ()>>, ()> (f=...)
at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/panic.rs:137
#30 0x000055893920f96d in std::thread::{impl#0}::spawn_unchecked_::{closure#1}<espanso::cli::worker::engine::funnel::init_and_spawn::{closure_env#0}, ()> () at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/thread/mod.rs:497
#31 0x000055893928d32e in core::ops::function::FnOnce::call_once<std::thread::{impl#0}::spawn_unchecked_::{closure_env#1}<espanso::cli::worker::engine::funnel::init_and_spawn::{closure_env#0}, ()>, ()> ()
at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/ops/function.rs:227
#32 0x000055893a5f79f3 in alloc::boxed::{impl#44}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/alloc/src/boxed.rs:1861
#33 alloc::boxed::{impl#44}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/alloc/src/boxed.rs:1861
#34 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#35 0x00007f86c582754d in start_thread (arg=<optimized out>) at pthread_create.c:442
#36 0x00007f86c58ac874 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100
(gdb)

[nhasian]

I wanted to compile from source on Linux for Wayland, but the instructions for Wayland just say to "To Do"

[hseliger]

I'm in no way an expert on the build system, but I managed to get it working under Wayland and Gnome (currently running Fedora 38/Gnome 44.2). I submitted a pull request to the espanso respository a while ago, but since I'm not part of the development team I have no idea what they are planning to with it. I may be too niche, as it won't fly under KDE, for instance.

If you want to give it a shot, try the wayland-info branch on hseliger/espanso (https://github.com/hseliger/espanso/tree/wayland-info).

Cheers,
Hendrik

[ZerkerEOD]

Any news on this. I was hoping to switch over to Manjaro KDE with Weyland support but the only thing that stopped me last year is that I have a VM that runs Espanso and want it to paste properly in the VM when using it so I disable it with the app-specific config on the host.

[neilzone]

Is there something specifically that is not working? I've been using espanso on Debian with Gnome, using the Wayland protocol, for a couple of years now. Do the build instructions work for you?

[ZerkerEOD]

It used to double paste, one with formating and one without. Coming from the hosts clipboard didn't format it like it should have when the one in the host formatted it correctly.