On Windows and on OSX with the default renderer, Sketch windows will open behind other windows

[hx2A]

On OSX Sketches using the default renderer to not get focus on sketch start. The Sketch window will be behind other windows, forcing me to hunt for it. The window will not have a dock icon I can click on to bring it to the front.

Steps to reproduce:

1. Run this Sketch:

def settings():
    py5.size(250, 250)

def draw():
    py5.rect(py5.mouse_x, py5.mouse_y, 10, 10)

2. Sketch window will not appear and will not have a dock icon. Need to minimize other windows to find it.
3. Curiously, if I run this in an interactive terminal (via jupyter console) and the Sketch is behind the terminal window, and the terminal window has focus and is partially transparent, I can still interact with the Sketch. The above Sketch will respond to mouse movements through the terminal window. This is only the case when the window first opens; if it gets focus and loses it, it won't happen anymore. It is as if the window knows it should have focus but does not.

[hx2A]

Commit 4cf255b is an ugly hack but alleviates the symptoms of this bug. That change needs to be removed when this bug gets a real fix.

[hx2A]

Windows had the same problem and the ugly hack was expanded to include Windows also. However, on Windows, when using the P2D renderer, forcing the window to be on top somehow triggers Processing to think the window was resized. When the window is resized, it re-applies the background color. This will overwrite any drawing activity that took place in setup.

[hx2A]

I investigated this further and I see that in Processing, the below code using an Open GL renderer has a problem on any OS:

PSurface surface;

void setup() {
  size(200, 200, P2D);
  fill(255);
  background(255, 0, 0);
  surface = getSurface();
}

void draw() {
  rect(mouseX, mouseY, 10, 10);
  if (frameCount % 500 == 0) {
    println("set always on top");
    surface.setAlwaysOnTop(true);
  }
}

The screen gets reset to the background color when setAlwaysOnTop gets called. But, py5 needs to make a call to setAlwaysOnTop on Windows because otherwise the Sketch window will be open behind other windows.

I dug into the Processing code and I highly suspect the bug is in Jogamp, not Processing. I doubt a fix will happen and most likely I will need to add a workaround to py5 to alleviate these symptoms.

[hx2A]

Continued from here: benfry/processing4#261

It seems like your use case is to change activate setAlwaysOnTop() after the program has been running for a while.

Well, that's just my example demo'ing the issue. If you draw to the window in setup() and also call setAlwaysOnTop(), the draw commands in setup will get overwritten by an extra call to background(). Perhaps I should have built my example around that. This is something that @tabreturn discovered last week.

In any case, here's where this is actually coming up:

https://github.com/hx2A/py5generator/blob/6ad2504b773567f86bb4307dc6d442949900fec9/py5_jar/src/py5/core/Sketch.java#L133

On Windows in py5, for whatever reason the Sketch window always opens behind other Windows. It's not Processing's fault, it must have something to do with the interaction between jpype and Processing. These kinds of idiosyncrasies appear now and then, and I need to write some code in py5 to smooth things out. Calling setAlwaysOnTop() twice like that before the call to the user's setup() method moves the window to the front without making it always on top, which might not be what the user wants. That code was added as a hack to alleviate this problem, and is not a proper fix.

I could add some code to cache the pixels between the end of setup() and the first call to draw(), but it feels like I am compounding a hack, and that's not the direction I want to go. I'll do it if I can't find another solution, but I don't have a pending release and would rather search for something better. Maybe there's a way to take out the setAlwaysOnTop() calls altogether and accomplish the same thing by accessing the native window object?

[jeremydouglass]

I see what you mean about "compounding a hack." I suspect that JOGL might possibly have been compounding a hack on native window managers:

https://jogamp.org/bugzilla/show_bug.cgi?id=1222

If the native manager only supports level assignment on window creation, then the only way that you can change the level is to recreate the window.

Maybe there's a way to take out the setAlwaysOnTop() calls altogether and accomplish the same thing by accessing the native window object?

Again, I haven't traced things through JOGL, but there is a chance that the content is disappearing precisely because they can't change the native window object, they can only destroy and recreate it. Which means, if they (JOGL) aren't copying your content forward when they do that, then you may have to....

[jeremydouglass]

Ha -- just noticed that the 2015 setAlwaysOnTop jogamp bug I found was filed by @codeanticode. All in the Processing family.

[hx2A]

If the native manager only supports level assignment on window creation, then the only way that you can change the level is to recreate the window.

Interesting, and great research! This is helpful.

Again, I haven't traced things through JOGL, but there is a chance that the content is disappearing precisely because they can't change the native window object, they can only destroy and recreate it. Which means, if they (JOGL) aren't copying your content forward when they do that, then you may have to....

I understand what you are saying. Caching the pixels between setup() and draw() might be the only option, but at least I would feel better about writing that code because I would know why I am doing it that way.

Ha -- just noticed that the 2015 setAlwaysOnTop jogamp bug I found was filed by @codeanticode. All in the Processing family.

That's beautiful!

[hx2A]

@jeremydouglass I fixed this. And the reality of what JOGL is doing is even uglier than what we previously thought.

Previously I was making two calls to setAlwaysOnTop(), one right after another.

        surface.setAlwaysOnTop(true);
        surface.setAlwaysOnTop(false);

I tried implementing code to capture the pixels after setup() and restore them before the first call to draw() but that didn't work. Eventually I figured out that if I made just one call to setAlwaysOnTop(), the capture / restore pair would work just fine. Apparently calling it twice like that triggers the window object to recreate the window twice, and it needs two trips through the animation loop to complete both window recreations. So now I need to make two separate calls to setAlwaysOnTop(), the first after setup(), and the second after the first call to draw(). See the deltas for 3cec106 for more details.

A side effect of this is that on Windows, while using an OpenGL renderer, if the user makes their own call to set_always_on_top(True) in their setup() method, it will get reset to False when frame_count == 1. Also, they'll need to do their own pixel capture / restore. This side effect might be fixable but not without compounding the hack even further, and I'm not going to go there.

While investigating this I observed that without any calls to setAlwaysOnTop() anywhere, the Sketch window will be second in the window ordering, right behind the browser. If it were to open someplace not covered by the browser, it will be visible, above other windows. Therefore I think the underlying problem isn't with py5 or Processing, and is instead something about the browser not giving up its position in the ordering.

[hx2A]

An alternative approach is to get the window handle and use a Python automation library to click on the window or somehow move it to the front. I don't like this idea so much either as it would add a new package dependency to py5 to facilitate a hack, but since a hack of some kind is necessary, I would consider it. Do you know of any good Python automation libraries that would be good candidate for this?

[jeremydouglass]

I fixed this.

woohoo!

make two separate calls to setAlwaysOnTop(), the first after setup(), and the second after the first call to draw()

Got it. A headache, but a solution.

One question -- where does the Windows+OpenGL surface.setAlwaysOnTop(false) happen? Should it be in restorePixels? I'm not seeing it in the 3cec106 deltas, but I may not be fully awake.

a Python automation library to click on the window or somehow move it to the front.

I'm not sure. Honestly, I'd support your instinct to avoid the added dependencies, partly because you'd probably need multiple OS-specific click libraries (unless you only need this for Windows, maybe pywinauto?), but especially because click-based solutions may be unreliable and may also introduce inconsistent focus problems (some paths / methods give the clicked window focus, some don't, results unreliable). But if there is a silver bullet out there, I hope you find it.

[hx2A]

Got it. A headache, but a solution.

Windows == headaches

One question -- where does the Windows+OpenGL surface.setAlwaysOnTop(false) happen? Should it be in restorePixels? I'm not seeing it in the 3cec106 deltas, but I may not be fully awake.

It is in capturePixels(). That method takes a boolean parameter and is called with true in setup() and false in draw(). I broke that out into a function in an attempt to make the hack less messy.

I'm not sure. Honestly, I'd support your instinct to avoid the added dependencies, partly because you'd probably need multiple OS-specific click libraries (unless you only need this for Windows, maybe pywinauto?), but especially because click-based solutions may be unreliable and may also introduce inconsistent focus problems (some paths / methods give the clicked window focus, some don't, results unreliable). But if there is a silver bullet out there, I hope you find it.

Right, I'd only need it for Windows, so perhaps it wouldn't be that bad. I agree that click based solutions are unreliable, but maybe I can do something if I get the handle id? I don't know. In any case, I'm happy with this solution for now and will re-address it again at a later date.