[Impeller] Jank when scrolling in ListView with impeller on iOS

[voxelbee]

Steps to reproduce

1. Create a new project use the code sample bellow and make sure you are running using impeller backend.
2. Scroll in the list and there will be a large amount of jank during scrolling.
3. Make sure to use a custom painter with a shader and layers as show in the example code. This makes the jank much more visible.

Code sample

Code sample

import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: ListView.builder(
        itemCount: 100,
        itemBuilder: (context, index) {
          return ListItemWidget(
            key: ValueKey<int>(index),
            index: index,
          );
        },
      ),
    );
  }
}

class ListItemWidget extends StatelessWidget {
  const ListItemWidget({required this.index, super.key});

  final int index;

  @override
  Widget build(BuildContext context) {
    return Padding(
      padding: const EdgeInsets.all(8),
      child: Painter(
        child: Card(
          child: ListTile(
            title: Text('Item $index'),
          ),
        ),
      ),
    );
  }
}

class Painter extends StatelessWidget {
  const Painter({required this.child, super.key});

  final Widget child;

  @override
  Widget build(BuildContext context) {
    return CustomPaint(
      painter: _CustomPainter(),
      child: child,
    );
  }
}

class _CustomPainter extends CustomPainter {
  _CustomPainter();

  static const List<Offset> bubbles = [
    Offset(0.2, 0.2),
    Offset(0.8, 0.8),
    Offset(0.5, 0.5),
  ];

  @override
  void paint(Canvas canvas, Size size) {
    final paint = Paint()
      ..style = PaintingStyle.fill
      ..filterQuality = FilterQuality.low
      ..maskFilter = const MaskFilter.blur(BlurStyle.normal, 35);

    final rect = (Offset.zero & size).inflate(40);

    canvas.saveLayer(rect, Paint());
    for (final bubble in bubbles) {
      final position = Offset(
        size.width * (1 - (bubble.dx * 0.4)),
        size.height * bubble.dy,
      );

      canvas.drawCircle(
        position,
        50,
        paint..color = Colors.red,
      );
    }

    canvas
      ..drawRect(
        rect,
        Paint()
          ..shader = const LinearGradient(
            begin: Alignment.topCenter,
            end: Alignment.bottomCenter,
            colors: [
              Colors.transparent,
              Colors.white,
              Colors.white,
              Colors.transparent,
            ],
            stops: [0.2, 0.3, 0.7, 0.8],
          ).createShader(rect)
          ..blendMode = BlendMode.dstIn,
      )
      ..drawRect(
        rect,
        Paint()
          ..shader = const LinearGradient(
            colors: [
              Colors.transparent,
              Colors.white,
              Colors.white,
              Colors.transparent,
            ],
            stops: [0.06, 0.1, 0.85, 0.94],
          ).createShader(rect)
          ..blendMode = BlendMode.dstIn,
      )
      ..restore();
  }

  @override
  bool shouldRepaint(covariant CustomPainter oldDelegate) {
    return false;
  }
}

Performance profiling on master channel

• The issue still persists on the master channel

Timeline Traces

Timeline Traces JSON

dart_devtools_2024-05-16_17 42 26.249.json.zip

Video demonstration

Video demonstration

[Upload media here]

What target platforms are you seeing this bug on?

iOS

OS/Browser name and version | Device information

iOS 17.4.1
IPhone 13 Pro Max

Does the problem occur on emulator/simulator as well as on physical devices?

Unknown

Is the problem only reproducible with Impeller?

Yes

Logs

Logs

[Paste your logs here]

Flutter Doctor output

Doctor output

[✓] Flutter (Channel master, 3.22.0-35.0.pre.35, on macOS 14.0 23A344 darwin-arm64, locale en-US)
    • Flutter version 3.22.0-35.0.pre.35 on channel master at /opt/homebrew/Caskroom/flutter/3.19.2/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision c719f03ded (18 minutes ago), 2024-05-16 18:12:33 +0200
    • Engine revision 460df6caef
    • Dart version 3.5.0 (build 3.5.0-160.0.dev)
    • DevTools version 2.36.0-dev.10

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
    • Android SDK at /Users/peytonhammersley/Library/Android/sdk
    • Platform android-34, build-tools 33.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15A507
    • CocoaPods version 1.15.2

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.87.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.86.0

[✓] VS Code (version 1.90.0-insider)
    • VS Code at /Applications/Visual Studio Code - Insiders.app/Contents
    • Flutter extension version 3.88.0

[✓] Connected device (2 available)
    • Peyton’s iPhone (mobile)        • 00008110-000A1C2811A3801E • ios    • iOS 17.4.1 21E236
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad     • darwin • macOS 14.0 23A344 darwin-arm64

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[jonahwilliams]

Debugging this a bit, the cost is evenly split between the rrect blur and the linear gradient shader. The ALU is limiting operations, which I'd expect for rrect blur but gradient is a bit surprising (well, not that surprising). So if we need an example of where some elbow grease is needed for rrect blur or gradient this is probably it.

[mark8044]

Ive been noticing increasing jank in list views, which was only very minor and so subtle as not noticeable in 3.10.x until now its more noticeable and mildly annoying in 3.22. I do have alot of gradients and blurs and images inside my listview

[jonahwilliams]

@mark8044 Without more information (i.e. a full bug report) we won't be able to say if that is the same issue as here.

[gaaclarke]

There is a fair bit of overdraw happening:

The team can work to make this faster but if this is representative of what you are really drawing I think the drawing could be simplified by drawing one transformed oval (instead of overlapping circles to make an oval) and then doing one dstin blend to clip it. The first one seems superfluous, in this example at least.

[jonahwilliams]

@gaaclarke if you look at the limiters though, its ALU ops and not memory writes. So while reducing overdraw will help its not the cause of the slowness w.r.t. skia

[gaaclarke]

@gaaclarke if you look at the limiters though, its ALU ops and not memory writes. So while reducing overdraw will help its not the cause of the slowness w.r.t. skia

Yea I saw that. Simplifying this to one oval and one gradient dstIn rect will reduce the math. It's not so much the overdraw in particular, but drawing 3 things to make one thing via overdraw is less efficient math-wise and memory-wise.

[voxelbee]

The team can work to make this faster but if this is representative of what you are really drawing I think the drawing could be simplified by drawing one transformed oval (instead of overlapping circles to make an oval) and then doing one dstin blend to clip it. The first one seems superfluous, in this example at least.

Yea well my actual code uses different colors for each of the circles so I was rendering them each separate. I guess I could optimize this a bit so that it renders one with transformations and a color blend instead? However, this wont still fix the issue that this is still incredibly slow. This doesn't seem to be the case when rendering the exact same UI with Skia.

[jonahwilliams]

Right, there may be ways to optimize the UI but we should still make a reasonable effort to be at least as fast as Skia.

[gaaclarke]

I landed one patch that reduces the arithmetic cycles for the linear gradient. It does also bump the required number of registers slightly though. Once it has merged into the framework it would be nice if the OP could give it a look.

I've spent some time looking into rrect_blur. The majority of the time comes from special functions (exp and sqrt). For the regular blur I was able to pre-calculate the gaussian blur coefficients on the CPU. It's difficult to do here since we are manipulating the blur based on the x and y values of the position, but I think it should be possible. I also briefly looked at our exp and sqrt usage.

• One exp is used for gaussian distribution calculation (in a loop)
• One exp is used for gaussian integration estimation. So even though exp is slow it is already being used as an optimization to avoid full integration.
• The sqrt is for finding distances for corners. I tried to look to see if we could perform the math squared space to avoid the sqrt. The distance is used to calculate the range over which we will integrate, which if we have a real gaussian distribution, overshooting it a bit might not be a problem if we wanted to estimage the sqrt. But since we are using an estimation for the integration I'm not so confident. (edit: relevant comment https://github.com/flutter/engine/blob/13cfb9004b2a5977543b26ab462e342cb0dc467b/impeller/entity/shaders/rrect_blur.frag#L78)

Jonah reminded me we should see where we can take advantage of lower precision. So I'm keeping an eye out for that. I might end up having to move the code to the cpu so I can get a better idea of what sort of numbers we're working with.

TLDR linear gradient should be faster, rrect_blur is a lot trickier to optimize.

[gaaclarke]

I transitioned rrect_blur to c++ so it can be examined for optimizations on the cpu: https://gist.github.com/gaaclarke/36ed9fad08f2a34a9230a61b328ea3cc

[gaaclarke]

I don't know if it's a fluke of my test, but the majority of the time sqrt in rrect_blur isn't used since we have an argument that is 1 or negative (which we throw away). I threw up a PR that will take advantage of that. The weird thing though is that when it isn't thrown away, only one ever argument is ever sent to sqrt. I'm not sure if we can pre-calculate that. It wasn't obvious that we could when i was staring at this.

[voxelbee]

I landed one patch that reduces the arithmetic cycles for the linear gradient. It does also bump the required number of registers slightly though. Once it has merged into the framework it would be nice if the OP could give it a look.

I just tested out the change it looks a bit better now then it was before the changes but it is still very far off from the performance with Skia. In Skia there isn't any frame spikes when scrolling and it runs really smooth. However, in Impeller there is still really visible frame spikes and jitter when scrolling. Also just posting flutter doctor output to check that I'm running the right version with the changes.

[✓] Flutter (Channel master, 3.23.0-9.0.pre.3, on macOS 14.5 23F79 darwin-arm64, locale en-US)
    • Flutter version 3.23.0-9.0.pre.3 on channel master at /opt/homebrew/Caskroom/flutter/3.19.2/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision a1a33e63b9 (6 hours ago), 2024-05-28 03:56:25 -0400
    • Engine revision b1751088c7
    • Dart version 3.5.0 (build 3.5.0-191.0.dev)
    • DevTools version 2.36.0-dev.10

[gaaclarke]

Thanks for checking, that version of flutter would have had the optimization at flutter/engine#53007. Possible further optimizations:

1. Conditionally doing sqrt in rrect_blur - we have a draft PR but the performance of gpu's surrounding branching is complicated
2. Eliminating the sqrt in rrect_blur - the cpu port of the shader seems to only call sqrt on a couple of numbers, seems like we could potentially pre-calculate that.
3. Moving the gaussian calculation (exp) to the cpu in rrect_blur - this is how I made the gaussian blur faster. I think it may be possible here too. It's a bit tricky since the fragment shader isn't reading from the whole curve like the gaussian blur.
4. Dropping fidelity - either in terms of floating point precision or render size.

Beyond that I'm not sure. The linear gradient could be optimized for the number of colors, but more variants creates performance problems elsewhere.

[jonahwilliams]

We can do much, much better with linear gradients: #148651

[jonahwilliams]

Numbers with flutter/engine#53166:

Skia: 3.5ms
Impeller ToT: 20ms
Impeller Patched: 11ms

So fixing gradients shaves off about half the time. To get the rest, we need to improve the performance of the rrect blur.

[jonahwilliams]

Ahh so for the rrect shader, the difference with Skia is that Skia has a specialized circle case which is much simpler.