Update interval when window is not visible

[matanox]

Hi,

I'm seeing that when my pyglet application's UI window is not visible (e.g. hidden by other full-screen windows) then the update interval temporarily goes up from the one set for the application, e.g.:

pyglet.clock.schedule_interval(self.update, 1/30)

― to almost exactly one second. For anywhere between 5 to 19 update cycles. After that amount of update cycles which varies from experiment to experiment, it resumes its normal rate. And the same when I click the window's minimize button and it minimizes ― several update cycles will last approximately exactly one second, before returning to the normal update rate.

The update rate reverts to normal after those several update cycles, while the window is still minimized or completely covered, the return to the normal update rate is not due to unhiding the window.

I'm just wondering if that's in correspondence to any known phenomena related to window managers, or whether off the top of your head you'd expect different behavior if upgrading from pyglet 2.0.5 to the latest. I recall that orthogonally, 2.0.9 had introduced new levers for finer control of redraws, not updates.

This is on Wayland.

Thanks!

---

• the yellows are the update-to-update interval, the infinitesimal blues at the bottom are the draw callback execution durations ― the latter do not change during the same time that the update-to-update interval spikes to one second.

• during the same time, the time spent inside my update callbacks doesn't change either:

---

As my application can be extremely sensitive to such large deviations, and can be assumed to be covered by other windows from time to time, I have instrumented it to easily view the times spent both within major functions, as well as the call-to-call intervals at which the callbacks are being called.

* in these runs, my update callback takes a little more than the requested udpate interval, which makes pyglet's scheduler invoke the next update callback as soon as the previous one finished (which I confirm with other measurements not included above).

[benmoran56]

My guess is that this is caused by the Window Mangager / Compositor de-prioritizing the Window context, and is expected behavior. What looks to be happening is that the internal calls to flip the Window are blocking for a period (which looks very synthetic).

For a test, and perhaps a solution, try to decouple the logic from the Window redrawing. Basically don't put anything inside of the on_draw event except for any movement code (taking delta time into account).

Also, if you have a small standalone example that I can test here, I can validate against my Gnome desktop for both Wayland and Xorg.

[matanox]

Thanks a lot for commenting @benmoran56 Indeed I have to work to reduce this to its minimally demonstrative code example, log and overlay the exact time of the hide and show events, which, pyglet aptly dispatches, and try to time the suggested internals or similar ones inspired by that thought ...

[matanox]

Well, zooming out to the bigger picture, this makes me realize I should probably consider splitting my update function so that some of its logic happens independently of pyglet's event loop, as it is managing things which should not hinge on how quickly displaying happens nor on whether display is at all enabled at a particular moment.

Things like video capture and inference. More precisely, to split the update callback into two pieces, keeping only one of them managed by pyglet's event loop and clock.

So I'll be possibly having:

• one update callback for pyglet to only prepare things for the next upcoming on_draw to keep the GUI smoothly going
• another update callback, or threaded code be it callback based or not, managing things which should always take place on a similar interval cadence, yet a cadence that cannot be directly affected by the windowing system or by the hiding of a window putting display logic into hibernation.

But then ― other than not creating graphical assets for pyglet from outside of the main thread, is there anything limiting the function of the pyglet event loop and window management ― when it is part of a multi-threaded python application?

It comes as no surprise that with python threads and the GIL, pyglet's code may wait for other threads' code execution as cPython is rotating which GIL-holding thread is running at any instance. If the overal system is not under load, pyglet's event loop may only deviate its callback scheduling by tiny amounts, and hopefully its proxying of keyboard and mouse IO handling doesn't break either ...

Or do you think that any of pyglet's event loop and event handling are written to contrary assumptions? Is there any test case for pyglet as part of a threaded python application or should I contribute one?

[benmoran56]

One thing that slipped my mind before, (I think you already know this) is that pyglet's clock (and scheduled events) will block. So the on_call/Window.flip running long will delay the execution of the next scheduled update. It seems like the only solution here will be threading. (That or switching to another desktop environment that doesn't have this behavior).

We don't have any examples of using threading with pyglet and, to be honest, don't really want to get into the landmine of trying to support user's threaded code. I'm probably going to revisit all of this whenever the no-GIL Python releases happens, but that's a ways off I guess.

For a test, maybe consider the following.
Using a Sprite as an example, you can't update it's position directly from a thread due to the OpenGL limitations.
However, you could make something like my_sprite.target_x, and update that from the thread. In the main thread, you then do the actual update: my_sprite.x = my_sprite.target_x.

[matanox]

Thanks for these thoughts. I'm actually using threads for file writing, IO-bound code and cpu-bound code that's releasing the GIL because it's not using python objects. It doesn't look like these are disturbing pyglet's event loop or its event handling ― no segfaults and no erratic clock behavior can be observed. I'm just not sure if that's luck or the event loop is really not prone to segfault in the presence of thread code running in parallel.

What kind of support for user threaded code would anyone miss or wish for?

In my code a cascade of processing happens on threads, both IO and also mostly C++ written cpu crunching which releases the GIL, but actually preparing drawable objects only on the main thread ― I don't see why this pattern should not be sufficient for any requirement ...

Obviously if the machine is under load the timing guarantees can become stressed too, I haven't systematically tested that, my application will break many of its real time guarantees if the machine is stressed, graceful degradation won't help this application much, so it's not much of a priority. Sure, I'll get to that one day.

[benmoran56]

Generally speaking threading is fine, but it's OpenGL itself that doesn't mesh well with it. There isn't anything we can do about that on the pyglet side. pyglet uses threading internally for other things, like audio and controllers, but you can't reliably update any OpenGL backed objects from those threads.

[matanox]

@benmoran56 thanks a lot for circling around to comment on this. this seems to explain why threading doesn't break the event loop when making sure to only address OpenGL from the main thread. Would you concur that this is an accurate way of putting it?

To be honest I wasn't sure from reviewing the event loop code, that it's not going to skew its timing guarantees when running as part of a threaded application; it seemed back then to assume that it's never going to be preempted from cpu execution but I could be wrong on it, it's been a while.

[benmoran56]

Yes, that's accurate. That's pretty much what most games or applications making use of OpenGL do.
That said, pyglet's internal threaded code is mainly calls to ctypes bound libraries, which do not hold the GIL.
If you run CPU bound Python code in a thread, then you'll be at the mercy of the interpreter.
The actual speed to which Python switches between GIL bound threads doesn't seem to be all that long, however. I came across this recently, which is supposed to be the target (but not guaranteed, apparantly).

>>> sys.getswitchinterval()
0.005

It would be interesting to do some tests on various platforms and see just what impact a 100% CPU thread would make.

[matanox]

Thanks, well obviously doing GIL-holding CPU bound work on a thread doesn't make sense for any python application. Just, sometimes one has some CPU work wrapping around proper GIL-releasing code that runs on a thread, so the more one has that in their application te more they would probably get a little more skew in event loop timing. So one may have to be very pedantic in not having such code.

But more crucially I still think that some event loop code may really crash on integrity issues as much as sys.getswitchinterval() enables things to happen on threads between its lines of code (half a milisecond is eternity in cpu terms) ― since it's not written defensively against that. Alluding to saying that calling any event loop api exposed by pyglet on threads can probably crash the event loop ― as the event loop api is presumably not written with any assumption of being called concurrently ― that was my take last reading it.

---

P.S. sys.getswitchinterval() must be python platform specific ― i.e. cpython v.s. other exotic ones, it also seems to be arbitrarily configurable in real time.

[benmoran56]

It's accurate to say that pyglet's clock and event dispatchers have not been heavily tested in multithreaded cases, and we don't really support that, but it will be revisited if/when the no-GIL Python builds make it out.

Events can be dispatched from threads, but not if the handlers are acting upon any OpenGL backed objects (sprites, text).
There is a way to dispatch events to the main thread. It's not publicly documented, however.

Normal event dispatching:

self.dispatch_event('event_name', *args)

Post event to main thread:

# self being the class instance that mixes in EventDispatcher.
pyglet.app.platform_event_loop.post_event(self, 'event_name', *args)