Add a new version of requestAnimationFrame for use with OffscreenCanvas

[junov]

The fundamental problems we need to solve are:

1. Window.requestAnimationFrame is not exposed in workers.
2. commit() may not be synchronized with a browsing context graphics update, so we need a version of requestAnimationFrame that is independent from the browsing context.

I have prepared an initial proposal that we can use as a starting point for discussion: https://wiki.whatwg.org/wiki/OffscreenCanvas.requestAnimationFrame

Any thoughts?

[junov]

Summoning @kenrussell @mephisto41 @grorg @toji

[kenrussell]

Nice work putting that together Justin. On an initial reading it sounds like a great start, and I don't immediately have any suggestions for improvement.

[domenic]

Given the open questions, what about an API like requestAnimationFrameAndCommit()?

[junov]

Given the open questions, what about an API like requestAnimationFrameAndCommit()?

Yes, I was also thinking we should merge the two, but I am not sure what the right form would be. An alternative would be to have only one commit method (as opposed to rAF vs non-rAF versions), and it would take an optional animation callback argument. Another possibility is to use promises:
ctx.commit().then(drawNextFrame);

These are all equivalent in terms of functionality. I'm just wondering which embodiment would be preferable.

[domenic]

It seems to me like merging them would be good to avoid the issues in open questions. I don't have a strong opinion on the particular design (or even on merging them; I'm really just an interested and hopefully helpful observer).

But I guess I am confused whether you want to run the code then commit, or commit then run the code. requestAnimationFrameAndCommit(codeToRun), or commit(codeToRunBeforeCommiting), would support the former. Whereas commit().then(codeToRun), or commit(codeToRunAfterCommiting) would work for the latter.

[domenic]

Is this issue a dupe of #2051 ?

[junov]

Is this issue a dupe of #2051 ?

D'Oh! I closed the other one.

[junov]

I guess I am confused whether you want to run the code then commit, or commit then run the code.

Hmmm. I was thinking commit then run the code for the next frame, which is sort of consistent with how window.rAF works. But both ways would work.

[junov]

Reposting comment by @toji here:

I don't think we want commit() to throw an exception, since one of the expected use cases is rendering to the headset AND mirroring to a PC monitor. rAF() is maybe a bit different, though I could maybe see an argument for having a main logic loop that looks like:

canvas.requestAnimationFrame(drawMirroredView); // Draws to main display at 60Hz
vrDisplay.requestAnimationFrame(drawVRView); // Draws to HMD at 90Hz

In a case like this the page would be drawing a different view of the scene (a third person view or "demo driver" view) to the main display, and drawing the first person view to the HMD. Thus far I've been coding such examples to draw to both within the single rAF, but that means that we're producing frames for the external display that are getting dropped on the floor. That's a somewhat odd pattern, though, and will produce an uneven GPU workload across frames, so I'm not sure if it's worth it.

[junov]

So in the case where we want the canvas to push a different view to the page than it does to the vr display, we would need separate commit mechanisms. I see that the WebVR spec already has its own commit-like thing: VRDisplay.submitFrame(). So I suppose OffscreenCanvas.commit() should just push a frame to the placeholder canvas, and should have no effect on any connected VRDisplays?

[toji]

That would work well for us. (And FWIW I'm proposing that we rename our submitFrame to commit as well, since they're really the same concept and just push the buffer to different places)

[domenic]

I guess I am confused whether you want to run the code then commit, or commit then run the code.

Hmmm. I was thinking commit then run the code for the next frame, which is sort of consistent with how window.rAF works. But both ways would work.

Well, requestAnimationFrame(cb) actually adds cb to the list of animation frame callbacks, and then the event loop first runs all those callbacks, then renders. So RAF is run code, possibly multiple times, then render.

Or did I misunderstand what you were thinking?

---

This brings up another important point. requestAnimationFrameAndCommit(cb), or some variation, is not very "composable" in the sense that multiple parts of the program cannot use it to add different callbacks. Whereas a separated requestAnimationFrame(cb) + commit() flow allows multiple calls to requestAnimationFrame, and thus multiple parts of the program to do that kind of work.

Is that kind of coordination aspect of RAF useful? Or do we envision people having a single loop? This kind of makes me lean back toward separate functions and just being sure to design the interaction correctly.