1544517 - Double-click (literal click! hw mouse) zooms in/out "feature" is a bad experience on android

Status: RESOLVED FIXED

(Core :: Panning and Zooming, enhancement, P3)

Description

[u580221]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

1. Double click a few times on a page (with actual hardware mouse)

Actual results:

Double-click (literal click! hw mouse) zooms in/out "feature" is a horrible experience on android. At best it gets in the way the rare times you click it, at worst with a laggy site firefox interprets clicks to make an unresponsive element work as "double tap" and zooms in and out all the time, making it just more infuriating. Please DISABLE this!

(I also need to reach for the screen to zoom back out since multizoom obviously doesn't work with the mouse. It's just an overall miserable experience)

Expected results:

Double click similarly to desktop doesn't do any zoom shenanigans

Comment 1

[Jan Henning [:JanH]]

Not sure whether there's any merit in considering to handle real mouse clicks any different there by default, but if you personally find it that bad, you should be able to turn this off via apz.allow_double_tap_zooming in about:config.

Comment 2

[Kartikaya Gupta (email:[e-mail])]

What kind of mouse are you using? And on what device? In general if the mouse is generating events with the right tool type we should be treating them as mouse, per the code here.

Comment 3

[Kartikaya Gupta (email:[e-mail])]

(To be clear, we don't do double-tap zoom if we get a double-click from a mouse. The bug is most likely that we're handling the mouse events as though they were touch, because the tool type check is failing)

Comment 4

[u580221]

Amazon Basics USB OTR adapter with Belkin USB 4 Port 2.0 Hub, with some Logitech USB corded office mouse (some cheap model, forgot which one)

if you personally find it that bad, you should be able to turn this off via apz.allow_double_tap_zooming in about:config.

That would disable it for tap as well, right? Not really a useful option

Comment 5

[Kartikaya Gupta (email:[e-mail])]

Randall, IIRC you had some Android setup with a mouse. Can you verify that double-clicking doesn't do a double-tap-zoom for you? Just want to make sure it's hardware specific as opposed to general bustage.

Assuming it's hardware-specific I can create a build with extra logging for jonas to run so we can figure out what the right condition is there.

Comment 6

[Randall Barker [:rbarker]]

I can confirm on my Pixel 2 with a bluetooth mouse, I can double click to zoom in. I verified it on release and beta by going to the full desktop page of amazon.com.

Comment 7

[u580221]

For what it's worth, I also noticed that text selection doesn't work, it just grab-drags like a finger instead. Just in case that's also because somehow firefox for android doesn't recognize the mouse as mouse

Comment 8

[Randall Barker [:rbarker]]

kats, sorry it's been a while since I worked on this. Treating the mouse as finger is unfortunately intentional. A while ago I made the change to make the mouse events actual mouse events on Android and we got a lot of push back from people using gray market android TV devices and they were no longer able to scroll because they didn't have a scroll wheel it had to be reverted. Let me see if I can find the bug...

Comment 9

[u580221]

Ok, but what about the double-click zoom then? Also, why not make the scrolling vs selection a setting? And wouldn't it be more progressive to enable text selection/proper mouse mode by default and build a blacklist of the broken devices, or is it not possible to identify them somehow by some id?

Comment 10

[u580221]

For what it's worth, there's Samsung Galaxy Dex and Google appears to experiment with a proper desktop mode in the Android 10 dev version. For these use cases having the mouse work properly will be somewhat relevant for a good experience

Comment 11

[Kartikaya Gupta (email:[e-mail])]

(In reply to Randall Barker [:rbarker] from comment #8)

Let me see if I can find the bug...

After some discussion on IRC and code archaeology, we found bug 1294707 is where this happened. I'm inclined to bring back the mouse behaviour behind a pref so that at least users who need it can flip the pref and get it.

Comment 12

[Kartikaya Gupta (email:[e-mail])]

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=e6225fa31ac1607974bf9985d1a25ea82b6e2101

Comment 13

[Kartikaya Gupta (email:[e-mail])]

Jonas, would you be able to try out the build from the try push above? You'll have to go into about:config and set the ui.android.mouse_as_touch pref to false, and that should take effect immediately and stop converting mouse inputs to touch inputs.

Here's a direct link to the ARM APK from the above run: https://queue.taskcluster.net/v1/task/I5lbUGuySOmNnniov-e9cw/runs/0/artifacts/public/build/target.apk

Comment 14

[u580221]

Yes, that fixes it: no more weird double click zoom, and even the text selection works!

I see in #1294707 there were some attempts to detect Android TV, and then apparently nobody followed through because it didn't reliably work. I'm wondering, can't you get Google to add something new in Android 10+ that returns whether a device is a TV and use that? Then at least in the future this could work properly on devices with Samsung Dex and Android 10's new experimental desktop mode, where it really doesn't make too much sense to treat any mouse as touch by default at all. Alternatively, have you ever considered a blacklist or something?

There has to be a better way than to either break this for all Android TVs, or break it for all desktop-like users the future will bring, and surely Google will at some point have to give this some thought with their own Chrome at some point, right? (given that in Android 10 they work on a true desktop mode) Maybe it might be time to poke them to figure out a proper solution

Comment 15

[u580221]

Okay, I just had an idea: as anyone ever asked Google to add a user-wide option for this in the input settings? That seems like the obvious solution as long as detection isn't feasible

Comment 16

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1544517 - Add a pref that controls conversion of mouse events to touch events. r=#geckoview-reviewers

For TV devices, it is useful to have mouse events automatically
interpreted as touch events, so we leave that as the default for now.
However, there are more and more Android devices which can be docked
or otherwise have real mice attached, and for those devices it's nice
to be able to treat the mouse inputs as a real mouse. This patch adds
a pref to control this behaviour.

Comment 17

[Kartikaya Gupta (email:[e-mail])]

Actually according to the docs, there is now a way to detect TV vs not: https://developer.android.com/training/tv/start/hardware.html#java

So I'll hook that up and allow the pref to be used as an override.

Comment 18

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1544517 - Switch mouse-as-touch behaviour to autodetect. r=#geckoview-reviewers

This changes the default behaviour to treat mouse as touch only on TV
devices.

Depends on D29188

Comment 19

[Randall Barker [:rbarker]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #18)

Created attachment 9061375 [details]
Bug 1544517 - Switch mouse-as-touch behaviour to autodetect. r=#geckoview-reviewers

This changes the default behaviour to treat mouse as touch only on TV
devices.

Depends on D29188

The problem is most Android TV devices are gray market and are just running the phone version of Android. I tried this the first time and unfortunately it didn't work.

Comment 20

[u580221]

I mean the settings already have a "Mouse/trackpad" section with a "Pointer speed" entry. I think it would really be a great idea, no matter what the default is, to get Google to add a "Emulate finger touch with mouse" option or similar. This would make this much less painful in case the default, whatever it is, ends up unsuitable for the given device

Comment 21

[u580221]

(also this might put the work of a reasonable default on Google. They might be more willing to maintain a black/whitelist of devices for a better default guess)

Comment 22

[Kartikaya Gupta (email:[e-mail])]

(In reply to Randall Barker [:rbarker] from comment #19)

The problem is most Android TV devices are gray market and are just running the phone version of Android. I tried this the first time and unfortunately it didn't work.

Ok, in that case we can just drop the second patch and leave the current behaviour as default with a pref override. It wasn't clear to me that this was what you tried in the original bug since you just posted an APK :)

Comment 23

[Kartikaya Gupta (email:[e-mail])]

(In reply to jonas from comment #20)

I mean the settings already have a "Mouse/trackpad" section with a "Pointer speed" entry. I think it would really be a great idea, no matter what the default is, to get Google to add a "Emulate finger touch with mouse" option or similar. This would make this much less painful in case the default, whatever it is, ends up unsuitable for the given device

Getting Google to do things is generally not easy. But you're welcome to file a bug against the Android bug tracker. https://source.android.com/setup/contribute/report-bugs

Comment 24

[u580221]

Wouldn't it make more sense if a Firefox dev filed this with the communicated intent to make use of it once it exists?

Comment 25

[Randall Barker [:rbarker]]

Adding an API wouldn't help since most of these TV devices run old versions of Android. I did everything I could think of the first time. There just doesn't seem to be a reliable API to detect device capabilities. I even tried querying the device to see if it was a mouse and if it had a scroll wheel. It didn't work.

Comment 26

[u580221]

What do you mean it wouldn't help? All the devices impacted by this with Android 10+ are modern and new, that is where the setting really matters. With older Android versions Firefox could just continue defaulting to always touch. To me that seems like the perfect solution, or am I missing something obvious?

But if I ask as a random user, I doubt they'll give it as much consideration compared to a developer asking

Comment 27

[u580221]

(Android 10+ is only now having that experimental desktop mode, so it seems like the big wave of devices where this matters is still to come. So that seems like the perfect time to see if this could be possibly added)

Comment 28

[Randall Barker [:rbarker]]

I wonder if it is worth leaving the default behavior for Fennec but having GeckoView default to supporting mice? @snorp any opinions on this?

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/491709a630ef
Add a pref that controls conversion of mouse events to touch events. r=geckoview-reviewers,rbarker

Comment 30

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/491709a630ef

Comment 31

[Narcis Beleuzu [:NarcisB]]

Backed out for Android geckoview failures.

Log for org.mozilla.geckoview.test.SessionLifecycleTest.collectOpen | status -2 : https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=243511335&repo=autoland&lineNumber=4070
Log for org.mozilla.geckoview.test.SessionLifecycleTest.collectClosed | status -2 : https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=243511374&repo=autoland&lineNumber=3811

Comment 32

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ad1f76c891a3
Backed out changeset 491709a630ef for geckoview failures

Comment 33

[u580221]

For what it's worth, I don't think it makes sense to treat this as fixed unless

1. it is enabled for a desktop-like Android device with hw mouse per default (which is probably not feasible at this point)

2. or there is either a preference setting in Firefox (you know, one users will actually find) or alternatively one in at least upcoming versions of Android in the system-wide settings (which I would propose to pursue) to toggle this on or off

Comment 34

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1cdb9beceade
Add a pref that controls conversion of mouse events to touch events. r=geckoview-reviewers,rbarker

Comment 35

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Randall Barker [:rbarker] from comment #28)

I wonder if it is worth leaving the default behavior for Fennec but having GeckoView default to supporting mice? @snorp any opinions on this?

Yeah, that's fine. Just flip this pref on in geckoview-prefs.js

Comment 36

[Kartikaya Gupta (email:[e-mail])]

Try push to make sure that doesn't regress anything: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=8243489e09f394b3872bb2ed967097455589dbda

Comment 37

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/1cdb9beceade

Comment 38

[u580221]

I don't think making a hidden option users most won't find is an ideal way to solve it. (Although of course it's quite cool to have it at all, thanks so much for that!)

Is someone interested in reopening this, or should I file a separate ticket to discuss the problem of this proper mouse handling not being enabled by default in fennec, and there not being any obvious way for a user to turn it on?

Comment 39

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1544517 - On GeckoView default to treating mouse inputs as mouse. r?snorp

Comment 40

[Kartikaya Gupta (email:[e-mail])]

(In reply to jonas from comment #38)

I don't think making a hidden option users most won't find is an ideal way to solve it. (Although of course it's quite cool to have it at all, thanks so much for that!)

You'd be surprised at how often users find obscure prefs - all it takes is one person to post it on a forum somewhere and it starts showing up in google searches. Nonetheless, this next patch will turn it on by default for GeckoView-based products, which are the future of Firefox on Android.

Is someone interested in reopening this, or should I file a separate ticket to discuss the problem of this proper mouse handling not being enabled by default in fennec, and there not being any obvious way for a user to turn it on?

This won't be fixed in Fennec because it's almost EOL now. The GeckoView-based products (Fenix in particular) will be replacing Fennec.

Comment 41

[Kartikaya Gupta (email:[e-mail])]

Reopening until the second patch lands.

Comment 42

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8eb5f9677c92
On GeckoView default to treating mouse inputs as mouse. r=snorp

Comment 43

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/8eb5f9677c92