Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

xorgrdp: XF86VidModeSetGamma() -> BadValue #271

Open
jknockel opened this issue Dec 31, 2023 · 3 comments · May be fixed by #272
Open

xorgrdp: XF86VidModeSetGamma() -> BadValue #271

jknockel opened this issue Dec 31, 2023 · 3 comments · May be fixed by #272
Labels
bug

Comments

@jknockel
Copy link

jknockel commented Dec 31, 2023 •
edited

xrdp version

0.9.17

Detailed xrdp version, build options

xrdp 0.9.17
  A Remote Desktop Protocol Server.
  Copyright (C) 2004-2020 Jay Sorg, Neutrino Labs, and all contributors.
  See https://github.com/neutrinolabs/xrdp for more information.

  Configure options:
      --enable-ipv6
      --enable-jpeg
      --enable-fuse
      --enable-rfxcodec
      --enable-opus
      --enable-painter
      --enable-vsock
      --build=x86_64-linux-gnu
      --prefix=/usr
      --includedir=${prefix}/include
      --mandir=${prefix}/share/man
      --infodir=${prefix}/share/info
      --sysconfdir=/etc
      --localstatedir=/var
      --disable-silent-rules
      --libdir=${prefix}/lib/x86_64-linux-gnu
      --libexecdir=${prefix}/lib/x86_64-linux-gnu
      --disable-maintainer-mode
      --disable-dependency-tracking
      --with-socketdir=/run/xrdp/sockdir
      build_alias=x86_64-linux-gnu
      CFLAGS=-g -O2 -ffile-prefix-map=/build/xrdp-Doma5Z/xrdp-0.9.17=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security 
      LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now -Wl,--as-needed
      CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2 -Wno-error=deprecated-declarations
      PKG_CONFIG_PATH=/build/xrdp-Doma5Z/xrdp-0.9.17/pkgconfig

  Compiled with OpenSSL 3.0.2 15 Mar 2022

Operating system & version

Ubuntu 22.04

Installation method

dnf / apt / zypper / pkg / etc

Which backend do you use?

xorgxrdp

What desktop environment do you use?

gnome-flashback

Environment xrdp running on

Physical or VM

What's your client?

No response

Area(s) with issue?

Graphic glitches, Other

Steps to reproduce

Compile and execute the following program in an xorgrdp session:

gamma.c.zip

✔️ Expected Behavior

Either the XF86VidMode extension not to be present or for the call to XF86VidModeSetGamma() to not deliver an asynchronous BadValue error.

❌ Actual Behavior

XF86VidModeSetGamma() mode crashes the program with a BadValue error.

Anything else?

Many X screensaver programs use XF86VidModeSetGamma() calls to implement a fade-to-black effect before starting the screensaver, but these appear to crash in an xorgrdp session with an asynchronous BadValue error. I couldn't find anything in terms of a specification on XF86VidModeSetGamma(), but these programs are written to not guard against an asynchronous error from calling XF86VidModeSetGamma() and, since they use Xlib, crash when they receive it. However, before calling XF86VidModeSetGamma(), they do check for the presence of the XF86VidMode extension and that it is at least version 2.0, but both of these conditions appear true in an xorgrdp session, and so they fatally call XF86VidModeSetGamma() anyways.
@jknockel jknockel added the bug label Dec 31, 2023
@matt335672
Copy link
Member

Thanks for this @jknockel

I'm moving this to the xorgxrdp repo where it can be addressed.

@matt335672 matt335672 transferred this issue from neutrinolabs/xrdp Jan 3, 2024
@matt335672 matt335672 linked a pull request Jan 3, 2024 that will close this issue
Open
@matt335672
Copy link
Member

BTW this is a great fault report - we don't get many test programs submitted.

By default the TigerVNC server (i.e. Xvnc) does not provide this extension.

Here's a bit of analysis:-

The current implementation of the server-side of XF86VidModeSetGamma() can be found here:-

https://cgit.freedesktop.org/xorg/xserver/tree/Xext/vidmode.c#n1391

From that we can see that for a virtual X server, the BadValue is going to be pretty inevitable.

The function pointer above is set up here for each screen:-

https://cgit.freedesktop.org/xorg/xserver/tree/hw/xfree86/common/xf86VidMode.c#n396

That's called from here:-

https://cgit.freedesktop.org/xorg/xserver/tree/hw/xfree86/common/xf86VidMode.c#n440

The best thing to do here is probably to disable the extension as it does not seem to server any useful purpose here. It's a fairly old part of Xorg as the name indicates, and it even has its own option DisableVidModeExtension to disable it.

I've raised a PR (above) to do this for review by the team. In the meantime, if you edit /etc/X11/xrdp/xorg.conf on your system and make the change in the PR this should do the trick for you.

Setting the DisableVidModeExtension option results in the internal server function xf86GetVidModeEnabled() returning false. The links above show that prevents the extension being initialised. The option serves no other function.

@jknockel
Copy link
Author

jknockel commented Jan 3, 2024

Thanks, adding Option "DisableVidModeExtension" "on" to the ServerFlags section prevents the crash on my end as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Disable XFree86-VidModeExtension for virtual server matt335672/xorgxrdp
2 participants
@jknockel @matt335672