Notify APZ of nested scrolling

RESOLVED FIXED in Firefox 67

Status

enhancement
P1
normal
RESOLVED FIXED
6 months ago
3 months ago

People

(Reporter: eeejay, Assigned: eeejay)

Tracking

unspecified
mozilla67
All
Android
Dependency tree / graph

Firefox Tracking Flags

(firefox67 fixed)

Details

Attachments

(4 attachments, 2 obsolete attachments)

Android has a setup for nested scrolling[1]. Via a CoordinatorLayout, descendant views negotiate top-down which one will consume a scroll action. In the case of Focus and android components, the toolbar and the GeckoView hand-off scrolling to each other. Currently, when scrolling down with a pan both the GV and the toolbar scroll, giving a weird accelerated effect for the first part of the pan until the toolbar goes away.

Ideally, there should only be one element scrolling at a time:
1. When the user starts panning down, the toolbar should scroll, and GeckoView should not (ie. it should remain static in relation the the receding toolbar)
2. When the toolbar is completely hidden, scrolling should be handed off to GeckoView.

This is consistent with how other native apps behave with a dynamic toolbar, including Chrome.

When I started fixing this in android components, I ran into two issues:
1. If we subtract the toolbar consumed y delta from the MotionEvent before dispatching it to GeckoView, APZ will trigger a long press because it will not register any relative movement from the down tap.
2. If the user continues a pan after the toolbar hides they will experience a brief hiccup in the scrolling because APZ needs to observe a minimal movement threshold to start a pan.

..so APZ needs to know when a touch is part of a nested scroll.

I also played around with preserving velocity from a fling when the toolbar is visible (eg. HandleDynamicToolbarMovement in Fennec), but I don't see any noticeable improvement in doing so, and it just adds complexity. If the participating scrolling sibling was higher than a toolbar there might be reason to revisit.

1. https://developer.android.com/reference/android/support/v4/view/NestedScrollingChild
Here is the fixed NestedGeckoView implementation that uses this flag:

https://github.com/eeejay/android-components/commit/72614c29e5f5ee4c8202b4194af9367d89bdc3e4

As me personally, and I'll give you an apk :)
Priority: -- → P1
Attachment #9032797 - Attachment is obsolete: true

I abandoned the previous changeset (with a comment of why). I have a new WIP that tracks absolute touch coordinates. This should allow us to use APZ's established heuristics and not reinvent them badly.

Comment on attachment 9036821 [details]
Bug 1515774 - Add screen offset to APZ for better touch interpretation.

I don't know how to do this in Phabricator.. just soliciting feedback before I clean this up more. AFAIK the two things I didn't resolve here are:

  1. Updating the velocity with the screen offsets. This works, often. But if the nested scroll snaps back the APZ still starts a fling and it looks bad. I kind of think this is low priority because the velocity is usually set fine when the user flings past the nested scroll.
  2. One finger pinch-zoom. It is just a bit chaotic with a dynamic toolbar, but not terrible. The GeckoView might need to be explicitly "pinned", that may require a bunch of messaging from APZ. Not a priority, imho.
Attachment #9036821 - Flags: feedback?(kats)
Attachment #9036821 - Flags: feedback?(geckoview)
Attachment #9036821 - Flags: feedback?(botond)

It would help understand the patch better if there was an explanation somewhere (here in the bug, in the commit message, in a code comment...) of what the offset being added represents :)

(In reply to Botond Ballo [:botond] from comment #7)

It would help understand the patch better if there was an explanation somewhere (here in the bug, in the commit message, in a code comment...) of what the offset being added represents :)

The offset represents the top-left screen coordinate of the GeckoView. If its parent is being actively scrolled it changes on each touch move (for example, with a disappearing toolbar, the y value will change.)

When we calculate distances, we need to factor those offsets in. We can't store the offset in the APZ's transform matrix because it would apply the same offset to all points uniformly and won't account for the position change.

Does that make sense? After sleeping on it, I think I am doing AsyncPanZoomController::TouchEventOffset wrong, but I get dizzy just thinking about all these coordinate spaces :)

The basic goal of the patch is to calculate pan distances and gesture detection based on the physical finger interaction on the screen, and not have it be relative to the GeckoView (which could be moving under the user's finger).

Comment on attachment 9036821 [details]
Bug 1515774 - Add screen offset to APZ for better touch interpretation.

(I wrote the feedback in Phabricator)

Attachment #9036821 - Flags: feedback?(botond)

We currently use ParentLayer pixels in GestureEventListener, it should
be Screen pixels because we care about physical distances and
thresholds, not layer-relative ones.

In Android the embedded GeckoView can be in a scrolling parent, and move
along with the user's finger. In order to intepret touch movements for
gestures, we need to account for the device position of each touch.

Depends on D17044

Attachment #9036821 - Flags: feedback?(kats)
Attachment #9036821 - Flags: feedback?(geckoview)
Attachment #9036821 - Attachment is obsolete: true

Comment 15

5 months ago
Pushed by eisaacson@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8e17cc234256
Use Screen pixels for gesture detection. r=botond
https://hg.mozilla.org/integration/autoland/rev/69c8787c67d6
Introduce mScreenOffset for pinch and multitouch events. r=botond
https://hg.mozilla.org/integration/autoland/rev/3d97f1e46c1a
Pass current GeckoView position with touch events to APZC. r=geckoview-reviewers,snorp
https://hg.mozilla.org/integration/autoland/rev/45c6f07160d9
Store window's screen offset in APZ and use it for gesture interpretation. r=botond
Attachment #9037719 - Attachment description: Bug 1515774 - Use Screen pixels for gesture detection. r?botond! → Bug 1515774 - Use Screen pixels for gesture detection. r=botond
Attachment #9037720 - Attachment description: Bug 1515774 - Introduce mScreenOffset for pinch and multitouch events. r?botond! → Bug 1515774 - Introduce mScreenOffset for pinch and multitouch events. r=botond
Attachment #9037721 - Attachment description: Bug 1515774 - Pass current GeckoView position with touch events to APZC. → Bug 1515774 - Pass current GeckoView position with touch events to APZC. r=snorp
Attachment #9037722 - Attachment description: Bug 1515774 - Store window's screen offset in APZ and use it for gesture interpretation. r?botond! → Bug 1515774 - Store window's screen offset in APZ and use it for gesture interpretation. r=botond

Comment 17

5 months ago
Pushed by eisaacson@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f6d635be8ffc
Use Screen pixels for gesture detection. r=botond
https://hg.mozilla.org/integration/autoland/rev/48a4ac1ba700
Introduce mScreenOffset for pinch and multitouch events. r=botond
https://hg.mozilla.org/integration/autoland/rev/2e07c6788179
Pass current GeckoView position with touch events to APZC. r=geckoview-reviewers,snorp
https://hg.mozilla.org/integration/autoland/rev/422db058e1cf
Store window's screen offset in APZ and use it for gesture interpretation. r=botond
Flags: needinfo?(eitan)
You need to log in before you can comment on or make changes to this bug.