1251617 - Scrolling using scrollbar thumb on long pages results in heavy checkerboarding

Status: NEW

(Core :: Panning and Zooming, defect, P3)

Description

[Jim Mathies [:jimm]]

Attachment: aboutsupport.txt

STR:

1) enable apz.drag.enabled
2) open a large DXR page and scroll using the thumb [1]

result: content paint fails to keep up with scroll position

[1] http://mxr.mozilla.org/mozilla-central/source/widget/gtk/nsWindow.cpp

Comment 1

[Botond Ballo [:botond]]

(In reply to Jim Mathies [:jimm] from comment #0)
> 2) open a large DXR page and scroll using the thumb [1]
> 
> [1] http://mxr.mozilla.org/mozilla-central/source/widget/gtk/nsWindow.cpp

(The link is to an MXR page, but I see the checkerboarding on both MXR and DXR.)

Comment 2

[Botond Ballo [:botond]]

This might just regular checkerboarding appearing worse due to the fact that on a very long page, scrollbar dragging allows you to scroll a lot more with the same mouse movement.

Comment 3

[Kevin Wern]

This is the only bug I could reproduce of those remaining for async-scrollbar-drag. However, the same also happens with touch scrolling, so this seems to be a universal APZ issue.

Based on a quick look around, it seems maybe this falls more cleanly under Bug 1256677. What do you think, Botond?

Comment 4

[Botond Ballo [:botond]]

(In reply to Kevin Wern from comment #3)
> This is the only bug I could reproduce of those remaining for
> async-scrollbar-drag. However, the same also happens with touch scrolling,
> so this seems to be a universal APZ issue.
> 
> Based on a quick look around, it seems maybe this falls more cleanly under
> Bug 1256677. What do you think, Botond?

Checkerboarding is indeed a universal APZ issue, and bug 1256677 tracks it in general.

This bug is on file because checkerboarding can be worse with scrollbar dragging than with other input methods. The reason is that with scrollbar dragging, you can scroll the content at an arbitrarily fast speed (since a fixed number of pixels in "scroll track space" can map onto an arbitrarily large number of pixels of scrolling, if the page is long enough).

Unfortunately, that also means checkerboarding is particularly difficult to solve for scrollbar dragging. We can't paint the entire page for long pages, and we can't really predict where the user will end up dragging to try to paint that region.

One possible mitigation may be to use "low-resolution painting", where we paint a larger region of the page than we otherwise would, at a lower resolution (think blurry text etc.) and show that during high-speed scrolling, then replace it with high-resolution painted content once the scrolling slows down. We have existing infrastructure to do this on Android (for fast flings) that we could try on desktop as well.

That said, this solution may be somewhat involved, and I don't think we should necessarily block enabling apz scrollbar dragging on it. We can enable it, see how widespread of an issue it is, and then decide if we'd like to employ the mitigation approach mentioned above, or do something else.

Comment 5

[Botond Ballo [:botond]]

(In reply to Kevin Wern from comment #3)
> This is the only bug I could reproduce of those remaining for
> async-scrollbar-drag.

I think we should be in good shape to enable scrollbar dragging on Nightly. If people end up running into those remaining hard-to-reproduce bugs, they will hopefully be able to provide URLs / testcases that will allow us to reproduce them.

Comment 6

[Botond Ballo [:botond]]

(In reply to Botond Ballo [:botond] from comment #5)
> I think we should be in good shape to enable scrollbar dragging on Nightly.
> If people end up running into those remaining hard-to-reproduce bugs, they
> will hopefully be able to provide URLs / testcases that will allow us to
> reproduce them.

Filed bug 1324117 for this.

Comment 7

[Zibi Braniecki [:zbraniecki][:gandalf]]

I experience this bug on https://metrics.mozilla.com/firefox-hardware-report/ (Dell XPS 13 9350, Linux, today's Nightly)

Here's a video of how it looks: https://us-next.owncube.com/index.php/s/pfIDvNmQnhMWBne

Comment 8

[Kartikaya Gupta (email:[e-mail])]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #7)
> Here's a video of how it looks:
> https://us-next.owncube.com/index.php/s/pfIDvNmQnhMWBne

Note that this bug is specifically about scrolling by dragging the scroll thumb. In your video it looks like you're scrolling via mousewheel or some other mechanism because the mouse cursor is over the web content and nowhere near the scrollbar.

That being said, yes, your video shows checkerboarding. See bug 1256677 for how we're tracking that.

Comment 9

[Zibi Braniecki [:zbraniecki][:gandalf]]

Yes, I was using a two-finger scroll on my trackpad.

> That being said, yes, your video shows checkerboarding. See bug 1256677 for how we're tracking that.

I'm not sure if that means I should file a new bug or is that already tracked?

Comment 10

[Kartikaya Gupta (email:[e-mail])]

(In reply to Zibi Braniecki [:gandalf][:zibi] from comment #9)
> I'm not sure if that means I should file a new bug or is that already
> tracked?

Based on the video you posted I would say it's already tracked. It didn't look like an "exceptional" case that would deserve a new bug and investigation.

Comment 11

[ovidiu boca[:Ovidiu]]

After a discussion with Kats on https://bugzilla.mozilla.org/show_bug.cgi?id=1349236#c7, I will add here a test case in order to verify this issue when the bug is fixed. 


Tested on Mac OS X 10.10, MacBook Pro 10.12, Windows 10x64 using Nightly 55.0a1(2017-03-21).

STR: 

1. Open Firefox and go to:
https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-linux-debug/1482891457/mozilla-inbound_ubuntu32_vm-debug_test-gtest-bm07-tests1-linux32-build277.txt.gz 
2. Drag the scrollbar(you can use the mouse/touchscreen) and move it fast up and down. 
 
Expected result: The text is properly displayed, without any delay.

Actual result: The text it has some delay in displaying the context.

Comment 13

[Botond Ballo [:botond]]

The example page from bug 1365246 is a good testcase for this.

Comment 15

[Mayank Bansal]

Attachment: prep.zip

This is another example that shows extremely bad checker-boarding when scrolling by dragging the scrollbar by mouse.
(This testcase is from some very old bugzilla bug, FWIW)

Comment 17

[Vitaly Kravchenko]

Would it be possible to keep the current image instead of painting element white in case it can't be rendered in time? That way the rendering is still async, but no blinking would happen.

Comment 18

[Botond Ballo [:botond]]

(In reply to Vitaly Kravchenko from comment #17)

Would it be possible to keep the current image instead of painting element white in case it can't be rendered in time? That way the rendering is still async, but no blinking would happen.

What do you envision the behaviour of the scrollbar thumb to be in such a case? I see two possibilities:

1. The scrollbar thumb position stays in sync with the scroll position, i.e. we delay updating it until the new content is painted. In this case, we basically lose the responsiveness gains from async scrolling, since nothing reacts to the user's mouse movement until the content is painted. (We used to have a pref that enabled this behaviour, added in bug 967502, but it was removed in bug 1514823.)

2. The scrollbar thumb position is updated right away (in sync with the mouse movement). The problem here is that now the scrollbar thumb position and the scroll position are not in sync, and this could be confusing for users.

Comment 19

[Vitaly Kravchenko]

I would expect the scrollbar thumb position to stay in sync with the mouse movement. But maybe instead of invalidating the rendering of the element and painting it white, if there's not enough time to render it before the next frame, it would be better to finish the rendering anyway and update the element a frame or two later, when the current render is done?

At the moment it's possible to scroll at just the right speed for invalidation to happen on every frame, so the element is effectively white until one stops scrolling. With the approach described above, the user would at least see some some updates in between, without any blinking or the element staying white the whole time, and without scrollbar thumb lagging the mouse cursor.

Comment 20

[Botond Ballo [:botond]]

That would be possible, but it's not clear that it would be an improvement to the user experience.

Suppose the user is trying to scroll to some specific content, so they are looking at the scrollable content as they are dragging the scrollbar, and will stop dragging once they see what they're looking for.

Suppose the content they're looking for is 40% of the way down the page. With the described change, it might be that we are showing the content at that position (40%) while the scroll position based on the scroll thumb is further down, say at 50%. If the user stops dragging at this time, the page will soon repaint to show the content at 50%, and the user will be confused because they were expecting to see the 40% content.

With the current behaviour, a user in this situation would naturally just slow down their drag to a speed where painting can keep up, and can stop at the desired location.

Comment 21

[Chris Lord [:cwiiis]]

In this particular case, I don't think any async behaviour makes sense personally - the only reasons you drag a scrollbar by the scrollbar thumb are to quickly get to particular places, or to scan through a document. In both situations, 'synchronous' behaviour makes sense - I think the best behaviour in thumb-dragging would be to reduce the displayport to just the visible area (expanded to tile boundaries, I suppose?) and scroll synchronously with the mouse. This increases the chance of not tanking the frame-rate but still provides the desired behaviour. I don't think a smooth screen update is the priority in this particular case like it is when scrolling with the mouse-wheel/touchpad/touch-screen. This is option number 1 in Botond's comment #18.

Perhaps to maintain some extra smoothness, this reduction of the displayport/sync painting could be triggered by scrolling beyond a particular delta, or when checkerboarding occurs (and the current behaviour restored when the rate of change of the scroll position goes back to a rate that can be kept up with)?

Comment 22

[Botond Ballo [:botond]]

(In reply to Chris Lord [:cwiiis] from comment #21)

In this particular case, I don't think any async behaviour makes sense personally - the only reasons you drag a scrollbar by the scrollbar thumb are to quickly get to particular places, or to scan through a document. In both situations, 'synchronous' behaviour makes sense - I think the best behaviour in thumb-dragging would be to reduce the displayport to just the visible area (expanded to tile boundaries, I suppose?) and scroll synchronously with the mouse. This increases the chance of not tanking the frame-rate but still provides the desired behaviour. I don't think a smooth screen update is the priority in this particular case like it is when scrolling with the mouse-wheel/touchpad/touch-screen. This is option number 1 in Botond's comment #18.

If that's the case, why did we spend all that effort on bug 1211610 and its dependencies? :)

Clearly, there was a desire to prioritize responsiveness here, and I believe responsive scrollbar dragging (and more generally "async scrolling everwhere") was even considered a point of differentiation from other browsers in the leadup to Firefox Quantum.

Comment 23

[Chris Lord [:cwiiis]]

(In reply to Botond Ballo [:botond] from comment #22)

If that's the case, why did we spend all that effort on bug 1211610 and its dependencies? :)

Clearly, there was a desire to prioritize responsiveness here, and I believe responsive scrollbar dragging (and more generally "async scrolling everwhere") was even considered a point of differentiation from other browsers in the leadup to Firefox Quantum.

Well, that's a good question :) Differentiation-wise, Windows 10 IE and Edge both beat Firefox to this by a long way... But that's besides the point. I think it's important to have the mechanics hooked up just to reduce the number of different ways to do one thing and increase maintainability, and I think async scrolling is fine if you're scrolling within the displayport and the browser is keeping up - but in this not-uncommon situation where it can't keep up, synchronous scrolling is then preferable (thus the heuristics I suggested in the second paragraph).

Comment 24

[Botond Ballo [:botond]]

It sounds like you're suggesting bringing back apz.allow_checkerboarding=false (and making it the default), together with some heuristics to suppress the displayport during periods when we're janking due to not being able to keep up.

That sounds like a significant and potentially controversial change, but I suppose we could implement it (without making it the default) and then ask UX and other stakeholders about input regarding whether it would be an improvement to the user experience.

Comment 25

[Botond Ballo [:botond]]

(Or perhaps you're suggesting doing this for scrollbar dragging only, and not other input methods, which would be a less controversial change, though a bit tricker to implement.)

Comment 26

[Chris Lord [:cwiiis]]

(In reply to Botond Ballo [:botond] from comment #25)

(Or perhaps you're suggesting doing this for scrollbar dragging only, and not other input methods, which would be a less controversial change, though a bit tricker to implement.)

mm, really just suggesting this for scrollbar dragging - I think there's already an expectation that checkerboarding is fine with other scrolling methods, and I think a small amount would be fine with scrollbar dragging too, but it's too easy on long, complex pages (at least on slower machines) to trigger a situation of not seeing any decent visual feedback for what's happening.

All this said, I don't think this is a high priority issue personally (with zero data to backup my feelings of course :)) and I would suppose a combination of webrender and constantly improving PC specs will probably obsolete the problem in 99% of cases.

Comment 27

[Botond Ballo [:botond]]

Thanks, I think we're largely on the same page.

I do plan to bring up checkerboarding mitigations, along with mitigations for other issues caused by not being able to paint at the frame rate (I'm thinking of bug 1367770 and dependencies) in planning discussions for future scrolling work.

Comment 30

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is relatively low, S3. However, the bug has 4 duplicates.
:botond, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Comment 31

[BugBot (nomail) [:suhaib / :marco/ :calixte]]

The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.