650339 - Allow access to UI on pages that consume touch events and prevent panning

Status: VERIFIED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Matt Brubeck (:mbrubeck)]

Problem:  When we implement touch events (bug 544614), then pages can prevent the user from panning.  This is intentional, to allow pages to consume drag gestures for their own purposes (like Google maps, painting apps, drag-and-drap UI, gaming).  However, it makes it harder for user to access Fennec's sidebars and address bar.

In bug 441590 we proposed keeping an area around the border of the screen that always allows panning.  There is a test implementation of this in bug 544614.  This works, but it can interfere somewhat with normal use of pages like Google Maps.  It's easy to accidentally swipe out the sidebars when trying to drag from one side to the other.  It would be beneficial to let the content consume touch events from the entire page, not just the interior.

Other mobile browsers have the same problem with touch events preventing UI panning, but not quite as badly because they only have top bars, not sidebars.  In Safari, the escape hatch is the system status bar at the top; tapping the bar causes the page to scroll to the top.  In Android, the escape hatch is the Menu/Search/Back keys.

The Menu/Search/Back keys also work as an escape hatch on Fennec, except they don't allow access to the sidebars.  This blocks tab switching, bookmarking, and the "forward" button.  If we make the Menu key show the sidebars, or add a "Tabs" menu command that caused the tab sidebar to slide out, then the system keys would allow most or all of these cases as well.  (As a bonus, this could also help with some of our existing problems with sidebar discoverability.)

Ben made a similar proposal on IRC: when pressing the menu button, both sidebars slide in just a little bit, and tapping on them or panning them makes them come out.

(In reply to bug 544614 comment #30)
> This is sorta like the borders idea, but could we drop a notification box (or
> maybe its called a doorhanger, I get confused with the terminology) down from
> the top that says something to the effect of "the page is intercepting drag
> events, touch here to return control to Fennec"?

Note that if we do this, then after the user returns control to Fennec, we'd also need a way for them to start interacting with the web page again.  (This is a true of other "modal" options that have been proposed too.)

Comment 1

[Wesley Johnston (:wesj)]

Attachment: Escape borders

This is a port of my previous patch from bug 544614. Also adds tests, although they seem a bit finicky to me. I am putting it up for reference and testing, but I'm going to look at making the sidebars pan in slightly when the menu button is pressed in a separate patch.

Again, this stacks on top of the patches in bug 544614 and 648573.

Comment 2

[Wesley Johnston (:wesj)]

Attachment: WIP Menu Patch

This patch pans in the sidebars slightly when you press the menu button. Tapping either or them will pan it into view.

I'm using moz-transform for this, and got way overexcited about it, wondering if I could do all of the sidebar panning this way. So this patch actually does that. The big caveat is that I am no longer panning the sidebars/urlbar as if they are next to the page, but instead I pan them over the content. For the sidebars I kinda like it. For the urlbar... maybe not.

Still an occasional problems here where the sidebars will pan away, but leave a very very very small strip showing on the edge and throwing off things, but good enough to play with. I put up a build at:

http://dl.dropbox.com/u/72157/fennec-sidebars.apk

for people to play with if they want to.

Comment 3

[Robert Kaiser]

(In reply to comment #2)
> This patch pans in the sidebars slightly when you press the menu button.

Is "the menu button" a hardware key? If so, what about devices/platforms that have none?

Comment 4

[Wesley Johnston (:wesj)]

Attachment: Escape Border Patch v1

Escape borders. This only sets one for the top of the page, but additionally will allow panning in sidebars if the urlbar is showing.

Comment 5

[Matt Brubeck (:mbrubeck)]

Comment on attachment 528690 [details] [diff] [review]
Escape Border Patch v1

Looks good to me.  It works, and does not interfere with normal use on Google Maps.  (I need more real-world example pages to test.)  Discoverability will be a problem, but I think this is true for most other solutions too - and on Android at least we also have more-discoverable access points like the Back/Menu/Search system keys.

Let's try this out and see what we think after a week or two.

>+    let bcr = browser.getBoundingClientRect();
>+    let rect = {};
>+    rect.left = Math.max(0, bcr.left);
>+    rect.right = Math.min(window.innerWidth, bcr.right);
>+    rect.top = Math.max(0, bcr.top);
>+    rect.bottom = Math.min(window.innerHeight, bcr.bottom);

Could be written:

  let rect = new Rect(0, 0, window.innerWidth, window.innerHeight);
  rect.restrictTo(Rect.fromRect(bcr));


>-ifndef ANDROID
>-ifndef MOZ_PLATFORM_MAEMO
>-_BROWSER_FILES +=  browser_sidebars.js
>-endif
>-endif

Are you intentionally getting rid of this test, or was this just for local debugging purposes?  (If you are getting rid of it, we need to delete the JS file itself, too.)

Comment 6

[Matt Brubeck (:mbrubeck)]

Comment on attachment 528690 [details] [diff] [review]
Escape Border Patch v1

Sorry, one more review comment:  Please adjust the safetyX and safetyY thresholds for the pixel density, like we do with browser.ui.touch.*  -- we want them to be the same physical size regardless of display density.

Comment 7

[Wesley Johnston (:wesj)]

Attachment: Escape Border Patch v2

Nits addressed. Defaults to half inch escape border on the top. I also renamed contentCanCaptureMouse to contentMightCaptureMouse because I was getting confused with it and canCancelPan.

Comment 8

[Madhava Enros [:madhava]]

What are the about:config prefs to play with the sizes of the escape borders?

Just to capture some discussion from IRC, 

madhava:   ok so:  escape hatch of last resort = menu button (already works)
madhava:   method for getting to side areas:  approach 1. menu+swipe;  approach 2. side border, possibly mimicking starting a swipe on the bezel
mbrubeck:  menu button is also good because it's the same thing you need to do to escape these pages in the stock Android browser.

Comment 9

[Wesley Johnston (:wesj)]

pushed: http://hg.mozilla.org/mozilla-central/rev/a28f4e72593a

prefs are dom.w3c_touch_events.safetyX and safetyY

Comment 10

[Wesley Johnston (:wesj)]

Also included a quick fix for elementFromPoint. r=mbrubeck on irc

Comment 11

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Verified:
hitting menu button and then panning will cause the sidebar to appear
holding the menu button and panning at the same time will cause the VKB to appear.

Comment 12

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Mozilla/5.0 (Android; Linux armv71; rv6.0a1) Gecko/20110506 Firefox/6.0a1 Fennec/6.0a1
Device: Droid 2 
OS: Android 2.2

Mozilla/5.0 (Android; Linux armv71; rv6.0a1) Gecko/20110506 Firefox/6.0a1 Fennec/6.0a1
Device: Thunderbolt
OS: Android 2.2