Double-click (literal click! hw mouse) zooms in/out "feature" is a bad experience on android

RESOLVED FIXED in Firefox 68

Status

()

enhancement
P3
normal
RESOLVED FIXED
3 months ago
3 months ago

People

(Reporter: jonas, Assigned: kats)

Tracking

66 Branch
mozilla68
All
Android
Points:
---

Firefox Tracking Flags

(firefox68 fixed)

Details

Attachments

(2 attachments, 1 obsolete attachment)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

  1. Double click a few times on a page (with actual hardware mouse)

Actual results:

Double-click (literal click! hw mouse) zooms in/out "feature" is a horrible experience on android. At best it gets in the way the rare times you click it, at worst with a laggy site firefox interprets clicks to make an unresponsive element work as "double tap" and zooms in and out all the time, making it just more infuriating. Please DISABLE this!

(I also need to reach for the screen to zoom back out since multizoom obviously doesn't work with the mouse. It's just an overall miserable experience)

Expected results:

Double click similarly to desktop doesn't do any zoom shenanigans

Not sure whether there's any merit in considering to handle real mouse clicks any different there by default, but if you personally find it that bad, you should be able to turn this off via apz.allow_double_tap_zooming in about:config.

Component: General → Panning and Zooming
OS: Unspecified → Android
Product: Firefox for Android → Core
Hardware: Unspecified → All
Version: Firefox 66 → 66 Branch

What kind of mouse are you using? And on what device? In general if the mouse is generating events with the right tool type we should be treating them as mouse, per the code here.

Flags: needinfo?(jonas)

(To be clear, we don't do double-tap zoom if we get a double-click from a mouse. The bug is most likely that we're handling the mouse events as though they were touch, because the tool type check is failing)

Amazon Basics USB OTR adapter with Belkin USB 4 Port 2.0 Hub, with some Logitech USB corded office mouse (some cheap model, forgot which one)

if you personally find it that bad, you should be able to turn this off via apz.allow_double_tap_zooming in about:config.

That would disable it for tap as well, right? Not really a useful option

Flags: needinfo?(jonas)

Randall, IIRC you had some Android setup with a mouse. Can you verify that double-clicking doesn't do a double-tap-zoom for you? Just want to make sure it's hardware specific as opposed to general bustage.

Assuming it's hardware-specific I can create a build with extra logging for jonas to run so we can figure out what the right condition is there.

Flags: needinfo?(rbarker)

I can confirm on my Pixel 2 with a bluetooth mouse, I can double click to zoom in. I verified it on release and beta by going to the full desktop page of amazon.com.

Flags: needinfo?(rbarker)

For what it's worth, I also noticed that text selection doesn't work, it just grab-drags like a finger instead. Just in case that's also because somehow firefox for android doesn't recognize the mouse as mouse

kats, sorry it's been a while since I worked on this. Treating the mouse as finger is unfortunately intentional. A while ago I made the change to make the mouse events actual mouse events on Android and we got a lot of push back from people using gray market android TV devices and they were no longer able to scroll because they didn't have a scroll wheel it had to be reverted. Let me see if I can find the bug...

Ok, but what about the double-click zoom then? Also, why not make the scrolling vs selection a setting? And wouldn't it be more progressive to enable text selection/proper mouse mode by default and build a blacklist of the broken devices, or is it not possible to identify them somehow by some id?

For what it's worth, there's Samsung Galaxy Dex and Google appears to experiment with a proper desktop mode in the Android 10 dev version. For these use cases having the mouse work properly will be somewhat relevant for a good experience

(In reply to Randall Barker [:rbarker] from comment #8)

Let me see if I can find the bug...

After some discussion on IRC and code archaeology, we found bug 1294707 is where this happened. I'm inclined to bring back the mouse behaviour behind a pref so that at least users who need it can flip the pref and get it.

Assignee: nobody → kats
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Type: defect → enhancement
Priority: -- → P3

Jonas, would you be able to try out the build from the try push above? You'll have to go into about:config and set the ui.android.mouse_as_touch pref to false, and that should take effect immediately and stop converting mouse inputs to touch inputs.

Here's a direct link to the ARM APK from the above run: https://queue.taskcluster.net/v1/task/I5lbUGuySOmNnniov-e9cw/runs/0/artifacts/public/build/target.apk

Flags: needinfo?(jonas)

Yes, that fixes it: no more weird double click zoom, and even the text selection works!

I see in #1294707 there were some attempts to detect Android TV, and then apparently nobody followed through because it didn't reliably work. I'm wondering, can't you get Google to add something new in Android 10+ that returns whether a device is a TV and use that? Then at least in the future this could work properly on devices with Samsung Dex and Android 10's new experimental desktop mode, where it really doesn't make too much sense to treat any mouse as touch by default at all. Alternatively, have you ever considered a blacklist or something?

There has to be a better way than to either break this for all Android TVs, or break it for all desktop-like users the future will bring, and surely Google will at some point have to give this some thought with their own Chrome at some point, right? (given that in Android 10 they work on a true desktop mode) Maybe it might be time to poke them to figure out a proper solution

Flags: needinfo?(jonas)

Okay, I just had an idea: as anyone ever asked Google to add a user-wide option for this in the input settings? That seems like the obvious solution as long as detection isn't feasible

For TV devices, it is useful to have mouse events automatically
interpreted as touch events, so we leave that as the default for now.
However, there are more and more Android devices which can be docked
or otherwise have real mice attached, and for those devices it's nice
to be able to treat the mouse inputs as a real mouse. This patch adds
a pref to control this behaviour.

Actually according to the docs, there is now a way to detect TV vs not: https://developer.android.com/training/tv/start/hardware.html#java

So I'll hook that up and allow the pref to be used as an override.

Attachment #9061340 - Attachment description: Bug 1544517 - Add a pref that prevents conversion of mouse events to touch events. r=#geckoview-reviewers → Bug 1544517 - Add a pref that controls conversion of mouse events to touch events. r=#geckoview-reviewers

This changes the default behaviour to treat mouse as touch only on TV
devices.

Depends on D29188

(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #18)

Created attachment 9061375 [details]
Bug 1544517 - Switch mouse-as-touch behaviour to autodetect. r=#geckoview-reviewers

This changes the default behaviour to treat mouse as touch only on TV
devices.

Depends on D29188

The problem is most Android TV devices are gray market and are just running the phone version of Android. I tried this the first time and unfortunately it didn't work.

I mean the settings already have a "Mouse/trackpad" section with a "Pointer speed" entry. I think it would really be a great idea, no matter what the default is, to get Google to add a "Emulate finger touch with mouse" option or similar. This would make this much less painful in case the default, whatever it is, ends up unsuitable for the given device

(also this might put the work of a reasonable default on Google. They might be more willing to maintain a black/whitelist of devices for a better default guess)

(In reply to Randall Barker [:rbarker] from comment #19)

The problem is most Android TV devices are gray market and are just running the phone version of Android. I tried this the first time and unfortunately it didn't work.

Ok, in that case we can just drop the second patch and leave the current behaviour as default with a pref override. It wasn't clear to me that this was what you tried in the original bug since you just posted an APK :)

(In reply to jonas from comment #20)

I mean the settings already have a "Mouse/trackpad" section with a "Pointer speed" entry. I think it would really be a great idea, no matter what the default is, to get Google to add a "Emulate finger touch with mouse" option or similar. This would make this much less painful in case the default, whatever it is, ends up unsuitable for the given device

Getting Google to do things is generally not easy. But you're welcome to file a bug against the Android bug tracker. https://source.android.com/setup/contribute/report-bugs

Wouldn't it make more sense if a Firefox dev filed this with the communicated intent to make use of it once it exists?

Adding an API wouldn't help since most of these TV devices run old versions of Android. I did everything I could think of the first time. There just doesn't seem to be a reliable API to detect device capabilities. I even tried querying the device to see if it was a mouse and if it had a scroll wheel. It didn't work.

What do you mean it wouldn't help? All the devices impacted by this with Android 10+ are modern and new, that is where the setting really matters. With older Android versions Firefox could just continue defaulting to always touch. To me that seems like the perfect solution, or am I missing something obvious?

But if I ask as a random user, I doubt they'll give it as much consideration compared to a developer asking

(Android 10+ is only now having that experimental desktop mode, so it seems like the big wave of devices where this matters is still to come. So that seems like the perfect time to see if this could be possibly added)

I wonder if it is worth leaving the default behavior for Fennec but having GeckoView default to supporting mice? @snorp any opinions on this?

Flags: needinfo?(snorp)
Attachment #9061375 - Attachment is obsolete: true
Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/491709a630ef
Add a pref that controls conversion of mouse events to touch events. r=geckoview-reviewers,rbarker
Status: ASSIGNED → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68

Backed out for Android geckoview failures.

Log for org.mozilla.geckoview.test.SessionLifecycleTest.collectOpen | status -2 : https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=243511335&repo=autoland&lineNumber=4070
Log for org.mozilla.geckoview.test.SessionLifecycleTest.collectClosed | status -2 : https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=243511374&repo=autoland&lineNumber=3811

Status: RESOLVED → REOPENED
Flags: needinfo?(kats)
Resolution: FIXED → ---
Target Milestone: mozilla68 → ---
Backout by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ad1f76c891a3
Backed out changeset 491709a630ef for geckoview failures

For what it's worth, I don't think it makes sense to treat this as fixed unless

  1. it is enabled for a desktop-like Android device with hw mouse per default (which is probably not feasible at this point)

  2. or there is either a preference setting in Firefox (you know, one users will actually find) or alternatively one in at least upcoming versions of Android in the system-wide settings (which I would propose to pursue) to toggle this on or off

Flags: needinfo?(kats)
Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1cdb9beceade
Add a pref that controls conversion of mouse events to touch events. r=geckoview-reviewers,rbarker

(In reply to Randall Barker [:rbarker] from comment #28)

I wonder if it is worth leaving the default behavior for Fennec but having GeckoView default to supporting mice? @snorp any opinions on this?

Yeah, that's fine. Just flip this pref on in geckoview-prefs.js

Flags: needinfo?(snorp)
Status: REOPENED → RESOLVED
Closed: 3 months ago3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68

I don't think making a hidden option users most won't find is an ideal way to solve it. (Although of course it's quite cool to have it at all, thanks so much for that!)

Is someone interested in reopening this, or should I file a separate ticket to discuss the problem of this proper mouse handling not being enabled by default in fennec, and there not being any obvious way for a user to turn it on?

(In reply to jonas from comment #38)

I don't think making a hidden option users most won't find is an ideal way to solve it. (Although of course it's quite cool to have it at all, thanks so much for that!)

You'd be surprised at how often users find obscure prefs - all it takes is one person to post it on a forum somewhere and it starts showing up in google searches. Nonetheless, this next patch will turn it on by default for GeckoView-based products, which are the future of Firefox on Android.

Is someone interested in reopening this, or should I file a separate ticket to discuss the problem of this proper mouse handling not being enabled by default in fennec, and there not being any obvious way for a user to turn it on?

This won't be fixed in Fennec because it's almost EOL now. The GeckoView-based products (Fenix in particular) will be replacing Fennec.

Reopening until the second patch lands.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8eb5f9677c92
On GeckoView default to treating mouse inputs as mouse. r=snorp
Status: REOPENED → RESOLVED
Closed: 3 months ago3 months ago
Resolution: --- → FIXED
Regressions: 1547839
You need to log in before you can comment on or make changes to this bug.