possible memory leak in ubuntu 22.04

[ExpandingMan]

On an older machine that is on ubuntu qtile looks quite a lot like it's experiencing some kind of memory leak and v0.22.1 did not resolve the issue. It seems to be loading up a few GB of memory plus quite a lot of cache (and for some reason I don't understand it seems like the system is being effected by cache getting filled). Restarting qtile in-place seems to help a little, but doesn't completely resolve the issue. This manifests itself as cursor updates getting incredibly sluggish (i.e. windows only recognize where the cursor is after a horrendous latency).

I see no sign of this issue whatsoever on my multiple Manjaro machines which are also running qtile. This leads me to believe that it's an ubuntu issue, probably a library dependency that needs to get patched.

Has anyone been experiencing anything similar?

[pawamoy]

I'm having issues on Arch Linux. Not sure it's Qtile's fault.

HTOP shows 5.2GB in the RES column, and 29.3GB in the VIRT column for the command Xorg -nolisten tcp :0 vt1 -keeptty -auth /tmp/serverauth.xxxxxxxxxx.

I installed xrestop as mentioned in this SO comment:

$ xrestop
xrestop - Display: localhost:0
          Monitoring 16 clients. XErrors: 0
          Pixmaps:   30360K total, Other:      27K total, All:   30387K total

res-base Wins  GCs Fnts Pxms ShmS Misc   Pxm mem  Other   Total   PID Identifier    
1200000     6   44    1   46    0  542     8352K     14K   8366K   654 qtile
0000000     2    0    2    0    0  128     5400K      5K   5405K 47917 <unknown>
0400000     1    1    0    5    0  103     5400K      2K   5402K   670 picom
1000000     0    0    0    1    0    0     5400K      0B   5400K   693 <unknown>
1800000     9    3    1    4    0   41     3092K      2K   3094K 250680 Firefox
1400000     5    2    0    2    1   41     2644K      1K   2645K 250249 terminator
0800000     4    2    0    0    1   18       36K    600B     36K   666 volumeicon
0600000     4    2    0    0    1   18       36K    600B     36K   665 NetworkManager Applet
1a00000     1    1    0    0    0    8        0B    240B    240B  1573 xdg-desktop-portal-gtk
0e00000     1    1    0    0    0    8        0B    240B    240B   694 polkit-gnome-authentication-agent-1
3c00000     2    1    0    0    0    5        0B    192B    192B 47917 Dunst
1c00000     1    1    0    0    0    1        0B     72B     72B 259685 xrestop
0a00000     0    1    0    0    0    2        0B     72B     72B   756 <unknown>
1600000     0    1    0    0    0    0        0B     24B     24B 250249 <unknown>
0200000     0    1    0    0    0    0        0B     24B     24B   639 <unknown>
0c00000     0    0    0    0    0    0        0B      0B      0B   699 <unknown>

Will now try to reboot and see these values right after boot.

[pawamoy]

The electron apps eating a ton of memory do not help (telegram, signal, elements, discord, vscode, ...).

[pawamoy]

Output of xrestop is the same after reboot, so I guess it's not Qtile's fault. I'll try to investigate further 🤷

[tych0]

Can you paste the output of top -n 1 -p $(pidof qtile)?

[elParaguayo]

But is this only on Ubuntu?

Do you see similar behaviour with the default config?

[tych0]

Yeah, I suppose we could be looking at two different bugs here :)

[pawamoy]

OK thanks! I'll try disabling whole bars of widgets to see if the issue still happens. Also, totally random, but I had this warning in logs:

dbus-next is not installed. Cannot run inhibitor process.

...so I installed python-dbus-next. Could this be related? (a simple no is fine, you don't need to elaborate ^^)

[pawamoy]

For some reason it looks like the stuff allocated by qtile increases a lot when I xset dpms force off. That fits with what I experienced since I have this new GPU: if I afk too long, then the system becomes unresponsive (no wake up), or if it at least wakes up, input devices do nothing. I have to force poweroff. No I know that this is because the memory is saturated. Need to know why allocations increase (or stuff is not released) when monitors shut off.

I was running this and forced monitor off:

% while :; do sleep 5; xrestop -m 1 -b | grep qtile -A16 | grep total | cut -d~ -f2 | numfmt --to=iec; done
...
8.2M
8.2M
35M
35M
62M
62M
70M
70M
...

(it shows total bytes taken by qtile in xrestop every 5 seconds)

[pawamoy]

Yep, every time:

...
70M
70M
70M  # force off
96M
96M
124M
124M
151M
151M
179M
187M  # wake up
187M
...

[tych0]

Could this be related?

No, I don't think it's related.

For some reason it looks like the stuff allocated by qtile increases a lot when I xset dpms force off.

huh, an interesting clue. Can you xtrace qtile while doing your while loop test?

[pawamoy]

I followed the docs and used exec xtrace /usr/bin/qtile start &>/home/pawamoy/qtile.log, but that file is only getting addr2line: /usr/bin/qtile: file format not recognized lines written to it.

[pawamoy]

Used exec xtrace /usr/bin/python /usr/bin/qtile start &>/home/pawamoy/qtile.log instead, and here's what I get (this is the whole file, it doesn't move):

% cat qtile.log 


X.Org X Server 1.21.1.11
X Protocol Version 11, Revision 0
Current Operating System: Linux corsair 6.7.5-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 17 Feb 2024 14:02:33 +0000 x86_64
Kernel command line: initrd=\intel-ucode.img initrd=\initramfs-linux.img root=PARTUUID=84599d5c-4885-4159-a7a3-2c4247fd020f rw
 
Current version of pixman: 0.43.2
	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/var/log/Xorg.0.log", Time: Wed Feb 21 17:14:37 2024
(==) Using config file: "/etc/X11/xorg.conf"
(==) Using config directory: "/etc/X11/xorg.conf.d"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Warning:          Could not resolve keysym XF86CameraAccessEnable
> Warning:          Could not resolve keysym XF86CameraAccessDisable
> Warning:          Could not resolve keysym XF86CameraAccessToggle
> Warning:          Could not resolve keysym XF86NextElement
> Warning:          Could not resolve keysym XF86PreviousElement
> Warning:          Could not resolve keysym XF86AutopilotEngageToggle
> Warning:          Could not resolve keysym XF86MarkWaypoint
> Warning:          Could not resolve keysym XF86Sos
> Warning:          Could not resolve keysym XF86NavChart
> Warning:          Could not resolve keysym XF86FishingChart
> Warning:          Could not resolve keysym XF86SingleRangeRadar
> Warning:          Could not resolve keysym XF86DualRangeRadar
> Warning:          Could not resolve keysym XF86RadarOverlay
> Warning:          Could not resolve keysym XF86TraditionalSonar
> Warning:          Could not resolve keysym XF86ClearvuSonar
> Warning:          Could not resolve keysym XF86SidevuSonar
> Warning:          Could not resolve keysym XF86NavInfo
Errors from xkbcomp are not fatal to the X server
Function              File                                                  Line
--------------------------------------------------------------------------------
Waiting for initial location to become available...
Location: 48.00 N, 7.00 E
discover_other_daemon: 1GNOME_KEYRING_CONTROL=/run/user/1000/keyring

I redirected from one layer above so it shows the output of commands run within .xinitrc. If I xtrace just the qtile command, it simply logs

Function              File                                                  Line
--------------------------------------------------------------------------------

Not sure what I'm doing wrong and how I could trace qtile correctly 😅

[pawamoy]

Any hint at where I should look for memory/pixmaps leaks in Qtile's code? Having to reboot each time I afk for half an hour is super annoying 🫠

Note again that it started happening when I changed my graphics card for a more recent NVidia.

[tych0]

I think the best thing would be to get xtrace working; then you can look for allocations that aren't freed. i don't really understand the bit about addr2line though (xtrace shouldn't be doing that; and it makes sense, /usr/bin/qtile is a python script, not a binary).

alternatively, probably somewhere in drawer.py is where the actual issue is.

[pawamoy]

I tried hard to get some logs from xtrace but it didn't work. If I pass it /usr/bin/qtile it doesn't recognize it, and if I pass it /usr/bin/python qtile it simply outputs nothing, even though qtile spawns and works correctly. If someone can confirm they can get logs from xtrace and qtile and share the exact command they run, I'd appreciate it.

Thanks, will look into drawer.py!

[tych0]

I have started seeing some leaks as well with more recent qtiles:

[797348.091572] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=snapd.service,mems_allowed=0,global_oom,task_memcg=/system.slice/lightdm.service,task=Xorg,pid=498167,uid=0
[797348.091607] Out of memory: Killed process 498167 (Xorg) total-vm:11781604kB, anon-rss:10186936kB, file-rss:1120kB, shmem-rss:590836kB, UID:0 pgtables:22268kB oom_score_adj:0

[tych0]

ok, it looks like this was a false alarm. qtile was not leaning resources, but another program I have running was.