591398 - Use three-finger gesture swipe to enter/exit Tab Candy

Status: RESOLVED FIXED

(Firefox :: General, defect, P1)

Description

[Aza Raskin [:aza]]

A simple change to firefox.js to allow using a gesture to get into/out of Tab Candy.

Comment 1

[Aza Raskin [:aza]]

Attachment: Changing the gesture meaning.

Comment 2

[Dão Gottwald [:dao]]

Please get this ui-reviewed.

Comment 3

[Justin Dolske [:Dolske]]

Comment on attachment 469927 [details] [diff] [review]
Changing the gesture meaning.

Yeah, this needs ui-review first. I already use the existing gesture a lot, seems like having it for page navigation is extremely useful.

Sadly (AIUI) OS X doesn't give us access to the 4-finger gestures, which would be a natural fit on OS X.

Comment 4

[Aza Raskin [:aza]]

Fair enough. I'll wait for Limi's review (thanks for pointing out the need, Dão!). I've always used the keyboard for jumping up and down in a page, never used the three finger swipe.

The main interaction I care about is the getting-to-Panorama gesture, and less the getting out again gesture (which brings you to the top of the page, which is by far more useful).

Comment 5

[Aza Raskin [:aza]]

Thus, I propose this behavior (of course, this bit-rots my first and only patch, naturally):

* Swipe Down always gets you into Panorama
* Swipe Up zooms back to the tab you were on, unless you have moved the mouse and are pointed at another tab
* Swipe Up when not in Panorama jumps to the top of the page.

Thus, the only collateral is jumping to the bottom of a page via a gesture, which I argue is not a super-high usage use-case.

Comment 6

[:Felipe Gomes (needinfo for replies!)]

Huh I thought that there could be some way to access 4-finger swipes on 10.6 but  I didn't find any discussions on an API to do that.

Two thoughts: there are the rotate gestures that are available but aren't mapped to anything. Perhaps they'd work well for TabCandy (not considering discoverability). Twist right to enter, twist left to exit (or twist in any direction would toggle). I think we tried to map these to next tab/previous tab at some point but people found it too sensitive. It would be a matter of experimenting and see how it works.

Also, if Swipe Up/Down is the way to go, why not simplify it like this: If you're already at the top of the page, swipe up would enter tabcandy. Otherwise, it works as is. Swipe down would always scroll the page because it's harder to tell when you're in the bottom of a page (personally I sometimes try to swipe down to check if I reached the end)

Comment 7

[Aza Raskin [:aza]]

The rotate gestures are generally harder to perform physically, so I'd like to stick to the swipe up/down gestures.

Consistency is key to having gestures feel reliable. So I'd like to stick with the proposal in comment 5.

Comment 8

[Alex Limi (:limi) — Firefox UX Team]

(In reply to comment #4)
> Fair enough. I'll wait for Limi's review (thanks for pointing out the need,
> Dão!). I've always used the keyboard for jumping up and down in a page, never
> used the three finger swipe.

It's established, and in use, so I think the four-finger swipe would be a much better fit here, since we're not using that already.

Trying to make three-finger swipe do different things depending on where you are on the page is going to cause mode errors.

I wouldn't do rotate, and even the current pinch-to-zoom gestures are extremely error-prone, and we should look into heightening the threshold for those (but that's a different issue, sorry :).

Comment 9

[Alex Limi (:limi) — Firefox UX Team]

(In reply to comment #6)
> Huh I thought that there could be some way to access 4-finger swipes on 10.6
> but  I didn't find any discussions on an API to do that.

BetterTouchTool seems to be able to get to the four-finger swipe, maybe email the author and ask how?

http://superuser.com/questions/53308/can-you-rebind-four-finger-swipe-in-new-macbooks/71387#71387

Comment 10

[Aza Raskin [:aza]]

Then four-finger swipe it is.

Comment 11

[Ed Lee :Mardak]

FYI, you can add this to about:config to test:

browser.gesture.swipe.up.shift = Browser:ToggleTabView

Comment 12

[Ed Lee :Mardak]

Tom Dyas implemented the original 3-finger swipe logic, so he might be able to comment on the 4-finger swipe.

Comment 13

[Ed Lee :Mardak]

Also, fyi, the prefs also support various combinations of shift/alt/ctrl/meta.. you could even use all 4 modifiers! ;)

browser.gesture.swipe.up.shift.alt.ctrl.meta -> Browser:ToggleTabView

Comment 14

[Jürgen Kreileder]

Isn't four-finger swiping bound to Exposé (down) and Show-Desktop (up) by default in 10.6?

Also, Google Chrome uses the three fingers gesture for its Tab Overview (enter = down, exit = up or down).  Would be nice if similar things worked similarly across browsers.

Comment 15

[Aza Raskin [:aza]]

I am once again advocating 3-finger swipe. Here is why:

* 4-finger swipe is in use by the OS
* 3-finger swipe I would argue isn't used that often currently and can be re-appropriated (need Test Pilot data)
* From a interaction hierarchy standpoint 1 finger means move the cursor over the page, 2 means move the page, 3 should mean move the tab, 4 means move the window. It doesn't make sense for 3 to also be at the page level.
* We shouldn't used a modifier key because it requires two hands (and using a gesture enables one-handed browsing in the first place)

Comment 16

[Tom Dyas]

(In reply to comment #12)
> Tom Dyas implemented the original 3-finger swipe logic, so he might be able to
> comment on the 4-finger swipe.

Cocoa only passes the three-finger swipe to applications via the swipeWithEvent: method of NSResponder. (The event passed in to that method has no information regarding how many fingers were involved in the swipe.)  So you are not going to be able to easily determine when a four-finger swipe has occurred using the documented Cocoa API.

On Mac OS X 10.6, you could try tracking the individual touches on multi-touch trackpads that support such tracking using the following methods of NSResponder: touchesBeganWithEvent:,touchesMovedWithEvent:, touchesCancelledWithEvent:, and touchesEndedWithEvent:.  However, not all trackpads support that and it is only a documented API on 10.6.  I am not sure if it is available or not on 10.5 as an undocumented API.

Regardless, the cocoa widget code only monitors for swipeWithEvent:, magnifyWithEvent:, and rotateWithEvent: gestures.  While we have DOM events now for multi-touch, it is for multi-touch on a touch display and not for a multi-touch trackpad.  Some thought would need to be done, I think, for how touch events should integrate into the browser when the touches are not performed on the display itself.  (Of course, that would be the subject for new bug.)

Link to the info:  http://developer.apple.com/library/mac/documentation/Cocoa/Reference/ApplicationKit/Classes/NSResponder_Class/Reference/Reference.html#//apple_ref/doc/uid/20000015-SW35

(In reply to comment #15)
> I am once again advocating 3-finger swipe. Here is why:
> 
> * 4-finger swipe is in use by the OS
> * 3-finger swipe I would argue isn't used that often currently and can be
> re-appropriated (need Test Pilot data)
> * From a interaction hierarchy standpoint 1 finger means move the cursor over
> the page, 2 means move the page, 3 should mean move the tab, 4 means move the
> window. It doesn't make sense for 3 to also be at the page level.
> * We shouldn't used a modifier key because it requires two hands (and using a
> gesture enables one-handed browsing in the first place)

You have my vote.

I added the top page / bottom page behavior for the three-finger swipes out of my personal need to jump to the bottom of certain large web pages faster.  I certainly did not study, and I do not believe any Mozilla people have truly studied, what the "proper" use for those two swipe gestures should be.

If changing the meaning of the up/down swipe gestures will bring us into parity with Google Chrome, I'm all for it.  And even more for it if it gives access to Tab Candy.

Comment 17

[Tom Dyas]

I would also add that, if this change is to be made, it is critical to make it  now in 4.0 because a major version bump is the time to break such user expectations if warranted and not later in the 4.x series itself.

Comment 18

[Alex Limi (:limi) — Firefox UX Team]

(In reply to comment #15)
> I am once again advocating 3-finger swipe. Here is why:
> 
> * 4-finger swipe is in use by the OS
> * 3-finger swipe I would argue isn't used that often currently and can be
> re-appropriated (need Test Pilot data)
> * From a interaction hierarchy standpoint 1 finger means move the cursor over
> the page, 2 means move the page, 3 should mean move the tab, 4 means move the
> window. It doesn't make sense for 3 to also be at the page level.
> * We shouldn't used a modifier key because it requires two hands (and using a
> gesture enables one-handed browsing in the first place)

Yeah, I think this makes sense. Probably have to re-train Beltzner's muscle memory ;) — but I think three fingers to activate Panorama makes sense.

Comment 19

[Ed Lee :Mardak]

Somewhat unrelated to this bug, but if we're doing 3 fingers = move the tab, should left/right be set to move to the previous and next tab instead of back/forward?

You can do this now through about:config:
browser.gesture.swipe.left = Browser:PrevTab
browser.gesture.swipe.right = Browser:NextTab

Comment 20

[Tom Dyas]

(In reply to comment #19)
> Somewhat unrelated to this bug, but if we're doing 3 fingers = move the tab,
> should left/right be set to move to the previous and next tab instead of
> back/forward?

Safari and Chrome both assign left/right swipe to back/forward.  For consistency, Firefox should keep those mappings as the default.

Comment 21

[Frank Yan (:fryn)]

(In reply to comment #5)
> * Swipe Up when not in Panorama jumps to the top of the page.
> 
> Thus, the only collateral is jumping to the bottom of a page via a gesture,
> which I argue is not a super-high usage use-case.

'Swipe Up to jump to top of page' makes the gestures asymmetric and more prone to mode error, imho.

You would also need a more complex value for browser.gesture.swipe.up.

Comment 22

[Aza Raskin [:aza]]

Attachment: Patch v2

This patch has the following implementation (ui+ by Limi):

* swipe down with three fingers shows Panorama
* swipe up with three fingers either
  - hides Panorama if in Panorama
  - scrolls to the top of the page if not

Comment 23

[Alex Limi (:limi) — Firefox UX Team]

Comment on attachment 477166 [details] [diff] [review]
Patch v2

Actually, I'm never in favor of overloading operations to mean two different things. I think three-finger swipe should be either Panorama or top/bottom page, but not both, depending on where you are.

How about using three-finger tap for Panorama instead? Since it doesn't need two different operations, that might be an option if we can detect it. (Don't know what our capabilities are here)

Comment 24

[Tom Dyas]

(In reply to comment #23)
> How about using three-finger tap for Panorama instead? Since it doesn't need
> two different operations, that might be an option if we can detect it. (Don't
> know what our capabilities are here)

Three-finger tap is not a viable option in my view currently for the following reasons:

1.  There is no DOM support to pass such a gesture up to the browser code from the widget level.

2.  You would only be able to detect a three-finger tap if the cocoa widget code implemented the touch tracking methods that are only available on Mac OS X 10.6 and not 10.5.  Gesture methods only exist for zooms, rotation, and three-finger swipes.

3.  Moreover, at least on Mac OS X, the gesture would be inconsistent with how single and double taps are interpreted as left and right clicks, respectively. A triple tap would not be as discoverable in my opinion for user as up and down swipes.

Comment 25

[Jürgen Kreileder]

Using three-finger tap wouldn't be that intuitive in my opinion.  The zoom-out/in animation of Panorama just fits better with a swipe gesture.
Exposé, which has similarities with Panorama, also uses a swipe gesture.  And there's Google Chrome which already uses 3-fingers down for its Tab Overview.

I'd just make 3-fingers up unbound outside of Panorama mode for now by default.

(A three-finger tap would be more something for a direct context sensitive action, like 'open link in new tab'.)

Comment 26

[Frank Yan (:fryn)]

> Actually, I'm never in favor of overloading operations to mean two different
> things. I think three-finger swipe should be either Panorama or top/bottom
> page, but not both, depending on where you are.

For reasons I stated in comment 21, I concur.

> How about using three-finger tap for Panorama instead? Since it doesn't need
> two different operations, that might be an option if we can detect it. (Don't
> know what our capabilities are here)

A tap is more like a click. Entering Panorama is like zooming out, which is more like the default 4-finger down _swipe_. I think that doing nothing on three-finger up swipe when not in Panorama works best for now.

FWIW, I don't know how often people do this, but I sometimes map three-finger tap/click to middle-click (which opens link in new tab, closes tab, etc.)

Comment 27

[Ian Gilman [:iangilman]]

Comment on attachment 477166 [details] [diff] [review]
Patch v2

Implementation looks fine, but sounds like it needs to be updated due to the continued UI discussion.

Comment 28

[Aza Raskin [:aza]]

Attachment: Patch v3

New functionality as per Limi:

* three-finger swipe down to enter Panorama
* three-finger swipe up to leave Panorama

Comment 29

[Ian Gilman [:iangilman]]

Comment on attachment 477707 [details] [diff] [review]
Patch v3

Looking good.

Comment 30

[Aza Raskin [:aza]]

Any word on the review?

Comment 31

[Mike Beltzner [:beltzner, not reading bugmail]]

This isn't a blocker, IMO.

Grudgingly I'll agree with the three-finger gesture changes. My only fear is that it's a pretty high-impact accidental thing to go into, but our exit-from-Panorama behaviour's pretty good.

For those like me who're upset by the change, the prefs are re-mappable in about:config, no worries.

Comment 32

[Aza Raskin [:aza]]

Adding this to b7 as per Beltzner's comment in IRC.

Comment 33

[Mike Beltzner [:beltzner, not reading bugmail]]

This stays betaN until we get a reviewed patch, then we can switch it to beta7 to get it in (mostly for bookkeeping reasons, we go to war with the system we have!)

Should be a simple review, though.

Comment 34

[Dietrich Ayala (:dietrich)]

Comment on attachment 477707 [details] [diff] [review]
Patch v3

r=me. as a followup, would be nice to get the other enter/exit code to use these commands instead of calling the methods directly.

Comment 35

[Justin Wood (:Callek)]

http://hg.mozilla.org/mozilla-central/rev/9f96a631ecca

(In reply to comment #34)
> r=me. as a followup, would be nice to get the other enter/exit code to use
> these commands instead of calling the methods directly.

I'll leave followup filing to someone here.

Comment 36

[Aza Raskin [:aza]]

Follow up bug here: https://bugzilla.mozilla.org/show_bug.cgi?id=600757

Comment 37

[Mike Beltzner [:beltzner, not reading bugmail]]

(In reply to comment #19)
> Somewhat unrelated to this bug, but if we're doing 3 fingers = move the tab,
> should left/right be set to move to the previous and next tab instead of
> back/forward?
> 
> You can do this now through about:config:
> browser.gesture.swipe.left = Browser:PrevTab
> browser.gesture.swipe.right = Browser:NextTab

No, those should match what's done in Safari.

Comment 38

[Boris Zbarsky [:bzbarsky]]

> Thus, the only collateral is jumping to the bottom of a page via a gesture,
> which I argue is not a super-high usage use-case.

This is the "I want to comment on this bug" use case!

Also, jumping to top of page via three-finger swipe no longer works either, as expected per comments here.  This is also a very common use case for me (most commonly with W3C specs so I can get to the "newest version" link, but also on many other pages.

Comment 39

[Mike Beltzner [:beltzner, not reading bugmail]]

Yes, we all agree that it's a very common use case with Mozilla Developers and Web Developers. Those people are also very likely to be people who'll know how to set their about:config prefs back to how they like them.

This bug is FIXED now; please only add comments if verifying or re-opening. We don't need a bunch of pile-on hatred or love. :)

Comment 40

[Aza Raskin [:aza]]

While I agree that being able to jump to the top of a page using 3-finger swipe is awesome, the UX team wasn't able to reach consensus there. See Limi's comment above. Patch v2 includes an implementation of that method.

Comment 41

[Boris Zbarsky [:bzbarsky]]

> Those people are also very likely to be people who'll know how
> to set their about:config prefs back to how they like them.

Only if we document that; adding dev-doc-needed.  And sorry about the comment, but sheppy needs to know what needs documenting here....

Comment 42

[Eric Shepherd [:sheppy]]

Documented:

https://developer.mozilla.org/en/Firefox_4_for_developers#Other_changes

Comment 43

[Michael]

Could an option be added in preferences to let users set the behavior they want without having to use about:config?

Comment 44

[Mike Beltzner [:beltzner, not reading bugmail]]

(In reply to comment #43)
> Could an option be added in preferences to let users set the behavior they want
> without having to use about:config?

No.

Comment 45

[Patrick]

backed out as part of bug 613909. Can be manually restored by using about:config using the prefs as shown in http://hg.mozilla.org/mozilla-central/rev/556d2c5bef08

Comment 46

[Tracy Walker [:tracy]]

I have long since been owner of this feature.  removing myself from the request field.