Workweek layers2

Servo workweek layers, CSS animation, and iframes

• zwarich: This is the open-ended graphics discussion. The most important one to make a decision on is iframes, because some iframe-related issues are blocking loading web pages. Martin, jdm, and I discussed the iframe issues yesterday. Now, the compositor knows about the frame tree. The constellation task, at the point where it's committed to the load, creates a frame tree and uses the SetIds message to set the layout. But, if you have an iframe that comes in later, the constellation will know about it but it doesn't tell the compositor about it. When the constellation then tries to SetClipRect, the compositor fails because it does not know about it. So the current design is not compatible with dynamic iframes. We discussed, in the short term, extending the model where the compositor knowing about the frame tree to support incremental updates. Then the constellation can tell the compositor about updates. That will allow us to load the pages we were using for performance tracking without crashing. Removing iframes dynamically would still be bad and cause task failure. Longer-term, the decision point is: who should know about the frame tree? We had an idea when rewriting the compositor that it should be as web-agnostic as possible, so it shouldn't have to know about frames and things. It should just know about scenes. We thought about having each pipeline have its own connection to the compositor. Then, within a layer hierarchy, you could insert the entire contents of another context. This is a similar model that other compositors have used, and means the constellation would no longer be the funnel to the compositor. The compositor would need to know about other pipelines and how they're related. I think we do plan to go in that direction long-term, but it would be good to wait until we have more of the document loading state machine done in the script task because currently the question of "when do we render things" was just larsberg trying not to crash, not a Real

• zwarich: Spec Implementation. With h5ever, we should be able to start implementing the standard.

• jack: So the compositor knows about the frame tree so that rogue hackers can't stuff in to composite. It wants to ensure that things from render tasks are valid...

• zwarich: There is nothing preventing compositor messages from being forged.

• pcwalton: Not security; more knowing what tab is current.

• jdm: That's PaintPermission

• jack: Obviously you could forge things, but want to not do things accidentally.

• zwarich: The root context is still controlled by the constellation. That said, if something in the root context's task is compromised, there is nothing we can do about it.

• jack: So, first, you just want incremental addition but without removal?

• zwarich: Yes, we'll get more sites working and more WPT tests passing.

• jack: Why not add incremental removal, too?

• zwarich: It's just more stuff to do and if it's not required for functionality, we should focus on the long-term design instead of more throwaway code. We can do that, if it turns out sites or WPT tests need it. The beauty of this system is that you can make graphics buffers less forgeable by using a mach port on OSX, DRI3 on Linux, etc. So it works if every pipeline is in a different process, which is also great.

• gw: Is DRI3 available?

• larsberg: My understanding from talking to the EFL people is that there are Wayland drivers and DRI3 support for everything but NVidia linux. Good embedded graphics driver support as well.

• kmc: EGL?

• larsberg: NVidia has some proposed standards changes to EGL they are working on with Khronos. I believe there's a beta, but we don't have access to it.

• kmc: So we can do all of what we want except for on some Linux hardware?

• pcwalton: Yes.

• zwarich: For iframes, I think that's a pretty good both short- and long-term story.

• mrobinson: Another topic is coordinate systems in layers. We have asked a lot about how to handle scale in the compositor. Now, we use the CoreAnimation model, where the scene has a scale (for pinch-zoom or HIDPI). When you walk through the layer tree, the layer may have contents that are also twice as big, then if so it will scale them down to the right resolution. Now, the scene has a scale and it looks at the size of the page in coordinates and makes a decision about how to scale those. There are alternatives. You could have the layers in scaled coordinates. So if you pinch-zoom you could resize the layers. The downside here is that you would have to walk the layer tree all the time, so the scene scale is a bit more natural. I think we've settled on our current approach because it's simpler (the scene scale).

• jack: Why are pinch-zoom and retina stuff the same?

• mrobinson: A retina display is sort of like pinch-zoom'ing the whole browser window.

• mbrubeck: You're increased the device pixels per CSS pixel but not changing how many CSS pixels wide the viewport is.

• jack: How does it work with high-res images?

• mrobinson: When painting a layer, you know what resolution it's at. As you do your paint, you know...

• mbrubeck: Scale the image based on device pixels, not page pixels.

• larsberg: I'm so glad those are separate types in servo

• zwarich: After pinch-zoom, do you reset the scale to 1.0, or does the layer tree always track the scale that you're pinch-zoom'd at?

• mbrubeck: In gecko, they have a temporary transform for when you're in the middle of the transform, and reset it to 1.0 when you finish the gesture.

• mrobinson: We do that right now, but it's on the tiles.

• mbrubeck: Always have a temporary situation where the scale that you renedered at is not the scale it's displayed at.

• mrobinson: Major difference is that if you pinch-zoom and it modifies the pinch-zoom scale and it knows it needs to repaint all of the tiles, in Servo it always reflects the global scale. When Servo walks down the layer tree, it knows what the tile is rendered at and can scale accordingly. It's an open question how to handle this. What we're doing right now is a lot easier to implement but much more confusing if you look at the code.

• larsberg: It seems sensible when you explain it, but it's definitely a bit buried in the code.

• jack: Short wiki page?

• larsberg: Ms2ger has been pushing us to put those docs in the source instead of wiki pages.

• jack: That's fine.

• mrobinson: Will do. I think we're converging on the right design. Also have an issue in the compositor with forming a hierarchy properly because the layers don't specify their parents. There is a layer that represents the main frame (has to be the first one!). Then there's a sub one, and all sibling are children. Based on some structure of the layer tree, I think there should be a more deeply nested structure. Right now, it's all just hacked to run. You can probably run into issues with iframes here.

• pcwalton: Going to need nested transforms.

• mrobinson: Good point. If we have a proper hierachy, we can think more about coordinate systems because layers are in their parent's coordinate systems. It's fine now because everything is so flat, but once it starts nesting...

• zwarich: Hopefully once we start nesting, the parent/child coordinate relationship will make more sense.

• mrobinson: Will require more layout changes, since right now all layers are in absolute coordinates. It will have to be relative.

• pcwalton: Should be fine; number of layers is small.

• zwarich: The problem with more layers is the graphics resources, not the coordinates.

• mrobinson: I was more worried for parallel layout.

• pcwalton: Can compute them absolute and do a sequential pass to fix up quickly.

• zwarich: CSS transforms and animations. Nothing is going in the wrong direction for this support. There is a distinction between compositor-side and layout animation. Layout ones have been the poor-performing version classically.

• pcwalton: I have some ideas for a timer that is run by the compositor (for knowledge of vsyncs)

• jdm: Gecko refresh driver?

• pcwalton: Yes, though it's sort of hacked on. Our compositor syncs to vsyncs by using swap buffers, which is what gecko would have done if it existed at the beginning. Idea's the same. The timer can fire and layout can do a timer update of the CSS animation. That can just poke a property in the flow tree and be quite efficient. Should work for all animatable properties, so long as they don't require changes in the flow tree.

• jdm: Does the flow tree need to be not-dirty?

• pcwalton: I don't think so. The important thing is that if you do window.getComputedStyle it won't reflect the current state of the animation.

• gw: Chrome and gecko report different values (start vs end?).

• zwarich: Chrome probably follows WebKit model.

• zwarich: Most important that the compositor for every frame is synchronized to some global monotonic time. If it were a display, you'd take the vsync time, then be e.g. 1/60th of a second ahead. Need everybody doing the animations, videos...

• larsberg: ANIMATED GIFS!

• zwarich: Clearly.

• pcwalton: Need to integrate with scrolling so we can render something without jank.

• zwarich: The decision of where to split between did you finish painting or not by the vsync is hard. Would like a control system to do it, but there's too much noise in the system.

• pcwalton: I experimented a bit.

• jdm: Would it make sense to drive flushing layout on this timer? That's what Gecko does. Instead of relayouting every time script does an update.

• pcwalton: Layout doesn't result in painting until the compositor asks for buffers, so I don't know if it matters.

• zwarich: The most sensitive timing decisions are all related to page load. e.g., trying to reduce the number of page loads. The script task may have to make that decision, actually.

• pcwalton: The Gecko thing and ours may be equivalent; layout may just have been done in advance, instead of doing layout and painting just in time. I think they end up morally equivalent. Do layout first and then wait for the vsync to come in and do the painting.

• zwarich: Another thing is when you have multiple pipelines with layers and animations, you need to synchronize them. They're all on the same timescale, but if a script adds animations to two separate frames with the same logical start time, you need teh compositor to see them as having the same start time. The way to do it is you need a concept of a fence where if script is running and making changes that can be reflected in multiple pipelines then when that occurs you need the layer changes caused by that script to be associated with a fence so the compositor knows that Pipelines A and B need to be synchronized. That will be tricky, and I assume that all browsers today just work by being sequential.

• pcwalton: Out of process iframes need it... but I think Chromium has been working on it for three years and still haven't shipped it.

• zwarich: And that's just for cross-origin.

• pcwalton: And just for a subset of sites: banks and google properties only. Too many processes otherwise.

• mbrubeck: With tabs, don't they start grouping things in the processes already?

• pcwalton: This is for security, though. And if there's a limit, it's trivial to just jam the the page with iframes and force the grouping. So there isn't a great solution here except for a whitelist.

• mbrubeck: But, those are already trusted sites...

• pcwalton: The threat model is evil.com embeds trusted.com in an iframe. They want trusted.com always separate to avoid exploits. This limitation stems from process overhead issues.

• zwarich: I'd hope Servo's long-term security model is to have everything run in the same address space. That SHOULD be the goal of a browser using a dedicated PL language.

• pcwalton: Yes, long-term.

• kmc: We do have Rust codegen bugs, etc.

• pcwalton: More worried about unsafe code, SM code, etc. It's a long-term project.

• kmc: Also poised to take advantage of sandboxing since we're message-based.

• pcwalton: I'd like to ensure we don't make choices that make sandboxing hard in the future.

• zwarich: If the goal is to sandbox everything that Chrome does, then there's less of a distinction...

• kmc: I'd like to make us as secure as possible with no sandboxing, and then sandbox anyway. Defense in depth.

• pcwalton: There's the iframe problem (evil.com embedding your bank in an iframe) is really bad.

• dherman: Especially since it's easy for evil.com to make it looks like the bank.

• kmc: Process per iframe on linux should be fine.

• larsberg: Worried about mobile, not desktop linux.

• pcwalton: Hopefully we can do both...

• zwarich: Sometimes sandboxing is hard and introduces new bugs.

• mbrubeck: If processes are the limit, we may just start blocking lots of cross-origin iframes. Are there that many cross-process iframes?

• larsberg: Ads.

• yichoi: Is it possible to make a thread sandboxed?

• pcwalton: No, because it's in the same address space, unfortunately.