702492 - Support built-in "scroll mode" in Gecko for touch screens

Status: RESOLVED FIXED

(Core :: DOM: UI Events & Focus Handling, defect)

Description

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

If you load a random scrollable page into a minimal embedding (say [1]) in Gecko on a device with a touch screen, and then try to scroll the page using normal touch gestures, it doesn't work.  The touch events are interpreted as mouse events in gecko, and the mouse events are interpreted as an attempt to select text.  That's pretty annoying.

So far, frontends built on Gecko have needed to implement their own touch-event handling as a wrapper in JS.  That's OK but not a great way to centralize scrolling code, since it's distributed among the frontends.  (The "smooth scrolling" code for desktop lives in gecko, by contrast.)  For the b2g case, this gets much more complicated because each launched app will have its own iframe that the app launcher can't peek into.  Further, the web browser app itself will have its own iframe that needs to be scrolled, and the web browser won't (initially) be able to inject code into the content iframe.

We can build up another system using frame scripts etc. like we do in fennec, but at this point it might be simpler to just assume we want scrolling by default in Gecko on touch screens, instead of the mostly-useless mouse events.  This is helpful long-term because it allows us to build async scrolling into Gecko too, rather than forcing the plethora of frontends to do it manually.  (That's proven to not be a good approach so far.  Mea culpa.)

So the concrete proposal here is
 - automatically assume "touch event mode" on touch screens
 - add a "default action" for touch events that interprets them as attempts to pan around, and eventually scroll the target scroll frame 

The panning physics can be pretty dumb for now.  In fact, it's probably better to save kinetic panning for a followup.

Frontends would still be able to preventDefault() on the touch events and implement their own scrolling behavior.  (We need this for the b2g home screen.)

This implementation needs to work across processes.  (Like default "synchronous" scrolling does in <browser remote>.)

[1] https://github.com/cgjones/mozilla-central/blob/master/b2g/chrome/content/shell.xul

Comment 1

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

I think bug 603008 is a prereq for this.  Please remove if that's not right.

Comment 2

[:Felipe Gomes (needinfo for replies!)]

(In reply to Chris Jones [:cjones] [:warhammer] from comment #0)
> Frontends would still be able to preventDefault() on the touch events and
> implement their own scrolling behavior.  (We need this for the b2g home
> screen.)

Note that this is the traditional design but it brings the implication that the event delivery must be synchronous, which makes the cross-process case more complex.
That is, if you're talking about the content itself controlling the behavior. Or maybe you meant the controller to be able to call preventDefault but not the content itself? (in which case that would probably be fine).

IIRC wesj and pcwalton were talking about this very issue some days ago on #developers. There are ways to mitigate the problem like only forcing synchronous delivery for the touchdown (which content would be crazy to decide between scroll and text selection mid-touchmove?), and only doing that for pages with touch listeners.

Comment 3

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to Felipe Gomes (:felipe) from comment #2)
> (In reply to Chris Jones [:cjones] [:warhammer] from comment #0)
> > Frontends would still be able to preventDefault() on the touch events and
> > implement their own scrolling behavior.  (We need this for the b2g home
> > screen.)
> 
> Note that this is the traditional design but it brings the implication that
> the event delivery must be synchronous, which makes the cross-process case
> more complex.
> That is, if you're talking about the content itself controlling the
> behavior. Or maybe you meant the controller to be able to call
> preventDefault but not the content itself? (in which case that would
> probably be fine).
> 

There are two cases we care about, I think
 (1) Browser frontend/chrome that wants to define its own non-default panning physics.  It can use a capturing listener and stopPropagation()/preventDefault() in the "frontend" to do that.  After the capture, it's up to the frontend to deliver the event however it chooses.  This is essentially what fennec does today.
 (2) Web content that listens to touch events to do *its* own panning.  In a frontend that wants (1) (like fennec), then it's already up to the frontend to allow content to intercept the touch event; gecko doesn't care because the frontend has stolen the event away during capture.  For frontends that don't need (1) (b2g browser), we let gecko handle the panning.

The simplifying assumption we can make is that there's either a capture listener that will *always* capture the touch event (1), or there's no capture listener at all.  With that assumption, we don't need to implement full DOM-event semantics, and the browser frontend doesn't need to block on content.  In neither case do we need a bubble-phase listener for the event in the browser frontend, so once the event reaches content, it can stay there.

Comment 4

[Wesley Johnston (:wesj)]

Since the move to native Fennec, we've had a lot more time to look at and start using native API's for scrolling (i.e. GestureDetector, PinchDetector, etc.). I would guess most platforms that support touch scrolling also support sending us similar events? Can we use those to tell Gecko to scroll in these cases instead?

Gecko will have to manage any displayport/css viewport issues which is a non-trivial problem. We may have to add some logic in widget code to detect whether a touch is a click (although android also provides some special hooks to help with that. Can we expect all touch platforms to do that too?)? Seems simpler to me. Cons are that B2G will have to write their own scroll vs click detection code in widget (but they're doing that here already), and it won't be easily available to other consumers if we want to port to future-touch-os which doesn't provide its own scrolling/click detection code.

Somewhere stechz was at one point looking to move a bunch of Fennec's scrolling code into toolkit. I'll try to dig up that bug today as well.

Comment 5

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to Wesley Johnston (:wesj) from comment #4)
> Since the move to native Fennec, we've had a lot more time to look at and
> start using native API's for scrolling (i.e. GestureDetector, PinchDetector,
> etc.). I would guess most platforms that support touch scrolling also
> support sending us similar events? Can we use those to tell Gecko to scroll
> in these cases instead?
> 

B2G (Gonk) won't have that.  Using platform events when available to trigger actions seems like a good idea, but unless the events have a vector associated with them, something will need to manage the physics.

> Gecko will have to manage any displayport/css viewport issues which is a
> non-trivial problem.

CSS viewport hackery is pretty simple, and orthogonal to scrolling.  Sub-frame displayport management is trickier but we don't need it for a "default" mode, initially.  That can come later.  Fennec can continue to manage those manually (although the new frontend isn't doing this anymore either).

> We may have to add some logic in widget code to detect
> whether a touch is a click (although android also provides some special
> hooks to help with that. Can we expect all touch platforms to do that too?)?

Nope :).

Just to be clear, I'm not proposing to move all of fennec's panning/zooming code into Gecko, in this bug: I'd just like to add something akin to a third scrolling mode for nsIFrameLoader.renderMode that does what's noted in comment 0.  Creating a Grand Unified Touch Scrolling mode, if we'd ever want to do that, would be off in the future.

Comment 6

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

mozbrowser and apzc got us this.