Scroll performance significantly degraded on iPhone X, Xs devices due to irregular input events delivery.

[wemped]

Myself and some others discussing in #22314 have noticed significant scroll performance issues on iPhone X,Xs devices with scrolling views.

Anecdotally it seems drag gestures are the main issue, while flings seem slightly better.
@coreysprague suggested it may be because these devices have a 120hz gesture sample rate (https://www.macworld.com/article/3235709/iphone-x-samples-touch-input-at-120hz-for-faster-smoother-response.html).

Just want to clarify that this does happen even when running in release mode.

Steps to Reproduce

Here's a tiny sample app where you can reproduce the scroll jitters and frame skips, just grab an iPhone X,Xs and run in release mode.

import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    const padding = EdgeInsets.symmetric(vertical: 10, horizontal: 10);

    return MaterialApp(
      home: Scaffold(
        body: ListView(
          children: <Widget>[
            Container(
              margin: padding,
              height: 200,
              color: Colors.grey,
            ),
            Container(
              margin: padding,
              height: 200,
              color: Colors.orange,
            ),
            Container(
              margin: padding,
              height: 200,
              color: Colors.grey,
            ),
            Container(
              margin: padding,
              height: 200,
              color: Colors.orange,
            ),
            Container(
              margin: padding,
              height: 200,
              color: Colors.grey,
            ),
            Container(
              margin: padding,
              height: 200,
              color: Colors.orange,
            ),
            Container(
              margin: padding,
              height: 200,
              color: Colors.grey,
            ),
            Container(
              margin: padding,
              height: 200,
              color: Colors.orange,
            ),
          ],
        ),
      ),
    );
  }
}

Logs

No errors in logs w/ flutter run --verbose

No issued found w/ flutter analyze

[✓] Flutter (Channel stable, v1.2.1, on Mac OS X 10.14.4 18E226, locale en-US)
• Flutter version 1.2.1 at /Users/dwempe/flutter
• Framework revision 8661d8a (8 weeks ago), 2019-02-14 19:19:53 -0800
• Engine revision 3757390
• Dart version 2.1.2 (build 2.1.2-dev.0.0 0a7dcf17eb)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
• Android SDK at /Users/dwempe/Library/Android/sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-28, build-tools 28.0.3
• Java binary at: /Applications/Android Studio 2.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
• All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 10.2)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 10.2, Build version 10E125
• ios-deploy 1.9.4
• CocoaPods version 1.5.3

[✓] Android Studio (version 3.3)
• Android Studio at /Applications/Android Studio 2.app/Contents
• Flutter plugin version 34.0.1
• Dart plugin version 182.5215
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)

[✓] VS Code (version 1.33.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 2.24.0

[✓] Connected device (2 available)
• Android SDK built for x86 • emulator-5554 • android-x86 • Android 8.0.0 (API 26)
(emulator)
• _________ iPhone • 00008020-000B25CE0298002E • ios • iOS 12.2

[bobmoff]

I tried the tiny sample app on my iPhone X and iPhone 5. I can verify that only on the iPhone X there are frequent stutters (lost frames) while drag-scrolling. As soon as I let go of the screen with finger and it only scrolls on inertia, it is fine.

There is also an increase of these stutters if I swipe up to app switch on iPhone X and then just taps the app again to resume it, and right when it has been resumed again I start drag scrolling. This might be related to this issue: #28113

[xster]

Wild guess but we seem to have seen a similar issue in #6135 only during touch events. We may need to gate the incoming event flow.

[liyuqian]

Thanks to @yunyu 's example app, I reproduced some stutters on iPhone X but not on iPhone 6S.
Attached is the trace with --trace-skia enabled (GPU thread frame time exceeds budget around 26.8s).

dart-timeline-2019-3-29.json.zip

Considering that it only happened on iPhoneX but not iPhone6S, I think it's related with the GPU hardware and the OpenGL driver of iPhone X. My hope is that Skia's Metal backend will eventually solve this kind of problem. So far, I'm not sure how Flutter can fix this without Skia's help.

@bsalomon (Skia's GPU lead): can you see something suspicious from the trace? Does Skia currently run tests on iPhone X? I seem to only find iPhone 6/7 on perf.skia.org . I guess as iPhone X gets more market share, such performance issues will become more and more critical. Also note that this performance issue only happened during first launches or app switches. So any performance tests that did a lot of warm up repetitions may not catch it.

[yunyu]

@liyuqian Just to note, this performance issue is also noticeable if you hold your finger down while scrolling (it doesn't occur when you fling by releasing your finger). Not sure if that is a separate issue.

[tvolkert]

@yunyu if you're seeing this on the simulator, that's a different issue that happens when keeping the finger down on non-Flutter simulator apps too, like the Settings app.

[yunyu]

@tvolkert I am seeing this on a physical device with the example app linked previously.

[msw333]

The effect is severe on iPhone X and minimal, but still observable, on iPad Pro (2. Gen.). Haven’t tested with other devices yet.

BTW: Mostly, the jittering is gone after 2-3 sec. But I even experienced times, where the jittering stays and can only be stopped with a restart of the app.

[bsalomon]

Sorry for slow response, just got back from vacation. I took a look at the trace. There is a crazy amount of time being spent in GrRenderTargetOpList::onExecute (31ms in one frame) but there isn't enough tracing detail to know what is actually happening in there. We'd need more detailed tracing events to get a better idea of where the time is going.

Side note: it's probably not too early to start trying to integrate the Metal backend. We're at the point where it draws almost everything correctly and performance is approx equal to GL with low hanging fruit still left to pick.

[1rgs]

@bsalomon what would be the best way to get more detailed tracing?

[frasza]

@bsalomon Any update on this matter perhaps from your side?

[bsalomon]

I'd try removing this check for android framework so that we can see which ops are taking a long time:

https://cs.chromium.org/chromium/src/third_party/skia/src/gpu/GrRenderTargetOpList.cpp?q=GrRenderTargetOpList.cpp&sq=package:chromium&g=0&l=499

[derolf]

I see the same on my iPhone X.

However, interestingly when I turn on screen recording, the jitter completely disappears and everything is super smooth.

I verified that multiple times.

[tvolkert]

I'd be interested to know if the same problems happen with the new Metal backend enabled.

@derolf @rahulgs12 @msw333 @yunyu @frasza @bobmoff @wemped are any of you willing to build a custom Flutter engine and try it out? Instructions are available at #18208 (comment)

[derolf]

@tvolkert I am using platform views. Will the metal backend work with them?

[derolf]

Btw another interesting observation: I don’t see jitter when in landscape mode.

[wemped]

@derolf I still experience jitter in landscape mode

[tvolkert]

I am using platform views. Will the metal backend work with them?

@derolf it should. If you're willing to give it an early try, we'd love the feedback on what works for you and what doesn't!

[derolf]

@tvolkert I’ll prolly give it a shot tomorrow.

Does someone else see the jitter going away when doing screen recording?

[coreysprague]

@derolf I tested the screen recording theory and still see jitter in my app.

[derolf]

@wuyugege

also you try this app:https://apps.apple.com/cn/app/cryptograph-designer-charts/id1375281233 ,
on the iphone x/xs. scroll on the home page of this app you will find this bug apparently.

I can confirm this app having the jank-disease on my iPhone Xs and that’s also the exact behavior I have on the app I am developing.

[liyuqian]

@derolf : this issue is specific to iPhone X/Xs. If you have some issues on iPhone XR, maybe it's better to leave comments on #32170 or create a new issue :)

[derolf]

@liyuqian sorry, it was an Xs. I edited my comment above.

[derolf]

I still have some questions:

• when should the PR appear in master channel?

• does the PR also fix janky kinetic scrolling?

• does the PR also fix janky scrolling that disappears at landscape or screen recording?

[liyuqian]

@derolf :

• when should the PR appear in master channel?

Unfortunately, the previous patch has been reverted. But I'm relanding it with some additional fixes: flutter/engine#12280

• does the PR also fix janky kinetic scrolling?

The kinetic scrolling shouldn't be affected by input events so there shouldn't be any jank even without the fix. Can you please attach more details of the jank, especially on how to reproduce it?

• does the PR also fix janky scrolling that disappears at landscape or screen recording?

I'm not sure how it could disappear. It seems to me that the jank will disappear in all cases after the patch.

(Reopen the issue due to the revert.)

[derolf]

@liyuqian please see #40042 (reported by me) and #38293 (might be related).

[christian-muertz]

Seems to be fixed on master now

[cbracken]

@liyuqian should this bug be re-closed?

[liyuqian]

Yes, the relaneded fix in the framework since b9a34de . Closing this issue.

[liyuqian]

Reopen for another revert...

[volskaya]

I'm observing the same irregular input delivery and dropped events on Android OnePlus 7 Pro. Alleviated greatly with cherry picking #36616 , tho Miss at most one frame for irregular scroll drag events is still an issue and is apparent even more so on its 90hz screen, as opposed to iPhone's 60hz.

@derolf reports above that the jitters are gone, when screen recording. I observe the same, while the screen recorder is locked at 60fps.

[liyuqian]

@volskaya : thank you for the report! I'm creating a new issue #41118 to track it separately.

[liyuqian]

The fix has been relanded in 4891e4a . Hopefully it won't be reverted this time.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.