Xcomposite window capture not working for windows on other workspaces

[T-X]

Problem Description

Capturing a window with OBS Studio's Xcomposite window capture or gstreamer ximagesrc are not working when the according window is on a workspace which is currently not viewed on any monitor. The cause is probably that Xmonad is a non-reparenting window manager?

Initially I had reported the issue at OBS here: obsproject/obs-studio#4966. But after debugging we concluded that it is likely something Xmonad would need to tackle, something OBS cannot do anything about.

Steps to Reproduce

OBS:

1. Start an application with a window in your current workspace (for example VLC)
2. Start OBS on the same workspace next to it
3. Create a scene in OBS
4. Add a source to the scene of type "Window Capture (Xcomposite)" in OBS
5. Select the VLC winodw for the window capture
   -> Video playback in VLC is captured fine in OBS
6. Then either move VLC to a workspace not used by a monitor or switch the layout to "Full" so that OBS covers up VLC
   -> VLC capture freezes in OBS

OBS window capture recovers when captured window becomes visible on a monitor again.

gstreamer:

1. Start an application with a window in your current workspace (for example VLC)
2. Get the xid of the VLC window with xwininfo
3. In the same workspace start gstreamer via:

$ sleep 3; timeout 20 gst-launch-1.0 ximagesrc xid=0x2200014 ! videoconvert ! queue ! x264enc qp-min=18 ! avimux ! filesink location=gstreamer-ximagesrc-3.avi
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...

4. Move VLC to a workspace not visible on any monitor
   -> gstreamer crashes as follows:

X Error of failed request:  BadMatch (invalid parameter attributes)
  Major opcode of failed request:  73 (X_GetImage)
  Serial number of failed request:  501
  Current serial number in output stream:  501

Configuration File

A minimal configuration to reproduce the problem. ewmh is needed for the OBS window capture (or "wmctl" for instance, too) to detect any window at all.

import XMonad
import XMonad.Hooks.EwmhDesktops
import XMonad.Layout.NoBorders

main :: IO ()
main = xmonad $ ewmh defaultConfig { layoutHook = noBorders Full }

Checklist

• I've read CONTRIBUTING.md

• I tested my configuration

  • With xmonad version 0.17.0 (from Debian Sid, 0.17.0-2)
  • With xmonad-contrib version XXX (commit XXX if using git)

[T-X]

From a user perspective, ideally xmonad would be able to detect automatically if an application tries to capture a window through xcomposite and would take the necessary steps to allow it (but don't know much about X11/xmonad, not sure if that'd be easily possible?).

But I'd also be curious about potential workarounds, to enable capturing a window that is not visible on a monitor.

[geekosaur]

This doesn't sound like something related to reparenting; it wouldn't be able to find the window at all in that case, regardless of visibility.

Does it work if you use a compositor like compton or picom?

[geekosaur]

You could also try running the server with backing store enabled, but I don't think recent Xorg guarantees that it works.

[liskin]

I think this is because xmonad unmaps windows of hidden workspaces whereas some other WMs (GNOME in particular) just move the windows to some coordinates where they won't be seen.

Fixing this would mean changing how xmonad core works, and may result in increased cpu/battery usage. (Not saying whether it should or shouldn't be done, just that it can't be worked around in contrib and that it's a substantial change in core.)

[slotThe]

It would also prevent things like X.L.Stoppable from working entirely.

[T-X]

@geekosaur

Does it work if you use a compositor like compton or picom?

I installed and tried picom here on Debian and just ran "picom" with its default configuration. That does not seem to make a difference.

You could also try running the server with backing store enabled, but I don't think recent Xorg guarantees that it works.

It seems that backing store is already enabled here on Debian Sid for me:

$ grep "Backing store" /var/log/Xorg.0.log
[    21.744] (==) modeset(0): Backing store enabled

---

The really hacky workaround I came up with and am currently using is this one:

• https://github.com/T-X/xmonad-config/tree/record-studio -> T-X/xmonad-config@bc31fd1

So with that I can use the shortcut Mod+Backspace to mark and toggle the currently focused workspace for recording on and off. And the hack I'm using is that I'm abusing my otherwise unused HDMI-1 port as a virtual screen which the marked workspace gets moved to ensure that OBS can keep recording it. And with xrandr I toggle this virtual and the real screen back and forth between a mirrored and split layout depending on whether I'm currently viewing the recorded workspace on my real screen or not.

Note though that this still sometimes leaks some frames from the new workspace though when switching the real screen away from the recorded workspace. So it's also a bit racy, couldn't find a proper fix for that yet.