Raster cost of certain flare animations are quite high

[mehmetf]

We have a customer who came up with a number of animations taking up nearly 40ms per frame to render.

I am attaching a sample animation, Skia trace done via DevTools and the SKP image which can be loaded to Skia Debugger. To quote one of the developers I talked to:

The Skia tracing and skp told us that there are many paths with cubic verbs. Those are usually quite expensive to rasterize, especially on lower-end devices. Rive may be able to reduce the Flare rasterization cost by grouping those paths into different RepaintBoundary. Then Flutter can then raster cache the path and not re-rasterize every path in every frame.

Skia SKP: flare_jank.skp.gz
DevTools Trace: dart_devtools_2020_9_10-1599771373430000.json.gz
Animation: groceries.flr.tar.gz

[mehmetf]

It is entirely possible that these animations are quite expensive due to the way they were designed. In that case, it would be great if you can point me to "performance best practices" that I can relay to designers so they are mindful when creating such animations.

[mehmetf]

@luigi-rosso FYI

[luigi-rosso]

Thanks @mehmetf. One thing that comes to mind, did you try disabling anti-aliasing for these? We have an option for that in the FlareActor and it usually helps significantly on older devices.

Adding different portions of the animations to a RepaintBoundary would be challenging. Possible but would also require breaking the renderer up such that it understands which items belong to which boundary. In the past we had requested for lower level access to SkSurfaces to handle this ourselves (it'd help from having to break the animation into separate widgets which would impact the Flare code pretty deeply), but I understood this wasn't feasible with Flutter's threaded rendering.

[mehmetf]

Thanks @luigi-rosso . Capturing a skp and analyzing it in Skia seems too much for a designer to do. Are there a series of best practices that highlight what type of animations might perform badly on lower powered phones? Perhaps a simulator at design time or some sort of stats report that shows how many complex paths it will end up drawing etc.

I will try to turn of anti-aliasing. Thanks for the tip.

[mehmetf]

@liyuqian FYI, see Luigi's comment above.

[liyuqian]

@luigi-rosso : I'd like to know more details about why it's infeasible to have Flare directly access lower level SkSurfaces with Flutter's threaded rendering. Is it because Texture widget may go out of sync with other widgets similar to the issue faced by PlatformView (embedded Google Map view, web view) where Flutter has to merge the platform thread and UI thread to ensure the synchronization of PlatformView and other widgets?

[luigi-rosso]

Hi @liyuqian, I'm not sure about the PlatformView as I haven't experimented with that but it does sound somewhat similar.

When we'd last looked at this we were hoping to be able to allocate an SkSurface, grab its canvas, draw into it, and then blit it back to the screen in the same synchronous rendering op and then keep a reference to it around to re-blit on further frames where it was still valid. It would require the Flutter code to be able to manage the lifecycle of the SkSurface (as opposed to saveLayer which needs to be re-issued every frame).

Because Flutter uses the PictureRecorder (I seem to remember it being really similar if not identical to the skp protocol) to serialize the command buffer for rendering (such that it can then be handed off to the rendering thread for execution), adding something like this would require serializing the creation, parameterization, and use of the SkSurface into that protocol which currently doesn't support it. And then exposing that to Flutter.

For non-realtime purposes doing a toImage on a Picture is something we experimented with to cache parts of the vector animation that do not change at all, however to update parts that sometimes change, we'd need frame sync which that doesn't guarantee (on top of the performance considerations of passing the entire surface's framebuffer to the cpu and back to the gpu as necessary). This is where having SkSurface to use somewhat like a render target in DX/FBO in GL is what we'd really need.

When I'd discussed this with the Skia team, the only real solution here would be a heavy-handed mod to the engine allowing Flutter code to run on the native rendering thread.

Hope that makes sense, would really love to find a solution for cacheable rendering surfaces managed by Flutter code. It would unlock a lot of interesting functionality for us, along with big performance wins.