Closed Bug 1556983 Opened 5 years ago Closed 3 years ago

@media (pointer) reports `fine` as the primary pointer on OnePlus 3T and Moto G5

Categories

(GeckoView :: General, defect, P3)

Product:
GeckoView
Component:
General
Version:
67 Branch
Platform:
Unspecified
Android
Type:
defect

Tracking

(firefox67 wontfix, firefox67.0.1 wontfix, firefox68 wontfix, firefox69 wontfix, firefox96 fixed)

Status:
RESOLVED FIXED
Milestone:
96 Branch
Tracking Flags:
Tracking Status
firefox67 --- wontfix
firefox67.0.1 --- wontfix
firefox68 --- wontfix
firefox69 --- wontfix
firefox96 --- fixed

People

(Reporter: lorenzo.stanco, Assigned: m_kato)

References

(
URL
)

Details

Attachments

(4 files)

Screenshot_20190605-110606.jpg
54.39 KB, image/jpeg
Details
Screenshot_20190605-110607.jpg
162.76 KB, image/jpeg
Details
Screenshot_20190605-112151.jpg
60.60 KB, image/jpeg
Details
Bug 1556983 - Primary pointer capability should be coarse when having touch screen. r=#geckoview-reviewers
48 bytes, text/x-phabricator-request
Details | Review

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

On OnePlus 3T (OxygenOS 5.0.8, Android 8.0.0), @media features like pointer and hover don't work as expected. It matches (pointer: fine) and (hover: hover), but it shouldn't, since it's a touch device with no mouse or pen attached.

For example, on MDN example page for @media pointer, the phone uses the (pointer: fine) rules and displays a small blue checkbox, but it should use (pointer: coarse) instead, and display a big red checkbox (see screenshot).

Actual results:

The issue occurs in both current Firefox 67.0 (all extensions disabled, also in incognito) and in Nightly 68.0a1 (fresh install), but not in Focus 8.0.9 and on Chrome, which is the default system browser (see screenshot). However, I think it's something related to the device, since I tried using Firefox on an Android Studio emulator, and everything work as expected.

Expected results:

Is there something I can do to help you debug the issue? Can we verify if this is an issue with this particular (albeit popular) device, or if it's a much broad issue?

Thanks.

Hi, I did not managed to reproduce with devices Samsung Galaxy S8+(Android 8.0.0), OnePlus 6T(Android 9.0.1) on builds Firefox Nightly 68.0a1(2019-06-05), Firefox Beta 68.0b7 and Firefox Release 67.0.
We do not own a OnePlus 3T(Android 8.0.0) - maybe is device and firmware specific.
This seems like a GeckoView issue, asking Chris to triage.

Flags: needinfo?(cpeterson)

@ Emilio: do you know why @media pointer uses fine rules (blue square in the test page) on some Android devices but coarse rules (red square) on other devices?

I can reproduce this bug on my Moto G5 (Android 8.1.0). I see the blue square in Firefox 67/68, Fenix, R-B, and Focus 8.0 (GV). I see the expected red square in Chrome and Focus 6.1.1 (WebView). Maye this is related to screen size?

If I open the test page in desktop Firefox's Responsive Design Mode (and then reload the page), the square changes from blue (fine rules) to red (course rules).

https://mdn.mozillademos.org/en-US/docs/Web/CSS/@media/pointer$samples/Example?revision=1468945

URL: https://mdn.mozillademos.org/en-US/d...
status-firefox67: --- → affected
status-firefox68: --- → affected
status-firefox69: --- → affected
Component: General → CSS Parsing and Computation
Flags: needinfo?(cpeterson) → needinfo?(emilio)
OS: Unspecified → Android
Product: Firefox for Android → Core
Whiteboard: [geckoview]
Version: Firefox 67 → 67 Branch

(In reply to Chris Peterson [:cpeterson] from comment #5)

@ Emilio: do you know why @media pointer uses fine rules (blue square in the test page) on some Android devices but coarse rules (red square) on other devices?

I can reproduce this bug on my Moto G5 (Android 8.1.0). I see the blue square in Firefox 67/68, Fenix, R-B, and Focus 8.0 (GV). I see the expected red square in Chrome and Focus 6.1.1 (WebView). Maye this is related to screen size?

This comes from java code:

If I open the test page in desktop Firefox's Responsive Design Mode (and then reload the page), the square changes from blue (fine rules) to red (course rules).

That's expected, since responsive design mode overrides it: https://searchfox.org/mozilla-central/rev/153172de0c5bfca31ef861bd8fc0995f44cada6a/layout/style/nsMediaFeatures.cpp#282

Hiro, you wrote this code, do you know why this may be happening?

Flags: needinfo?(emilio) → needinfo?(hikezoe)

The best guess I can tell is that such devices have multiple pointer devices and report a fine-grained type device prior to the coarse one.

Chris, what is the result on this test page, https://hiikezoe.github.io/interaction-media-features.html? The page also reports any-pointer result on the device, so if any-pointer has both of fine and coarse, it must be the priority issue.

Also note that, IIRC Chrome has a bug on the media query (I did report it to them), but I don't recall exactly.

Flags: needinfo?(hikezoe)

Chris ^

Flags: needinfo?(cpeterson)

Oh wait, I was misreading comment 5. It happens only GeckoView?

(In reply to Hiroyuki Ikezoe (:hiro) from comment #7)

The best guess I can tell is that such devices have multiple pointer devices and report a fine-grained type device prior to the coarse one.

what is the result on this test page, https://hiikezoe.github.io/interaction-media-features.html?

On my Moto G5, Chrome and Fennec report different values for pointer:

Chrome reports:

pointer: coarse
any-pointer: fine
any-pointer: coarse
hover: hover
any-hover: hover

Fennec 67 reports:

pointer: fine
any-pointer: fine
any-pointer: coarse
hover: hover
any-hover: hover

Oh wait, I was misreading comment 5. It happens only GeckoView?

I can reproduce the problem in both Fennec and GeckoView (Fenix and Reference Browser) on my Moto G5. Diana might have meant "Gecko" when she wrote "GeckoView". She wasn't able to reproduce the problem on the devices she had, so the problem seems to be device-specific.

Flags: needinfo?(cpeterson)

Thank you Chris, the result totally makes sense to me. On Chrome if there are both of fine and coarse type pointers on the device, they always report the coarse is the primary pointer, whereas we report the first pointer which founded by an Android API. I believe our way is more reasonable and I did actually test the behavior and requested QA to test it such as

  1. a mobile phone with mouse connected in the first place (i.e. with a mouse when power on)
  2. a mobile phone with a mouse which was connected after boot

In case of 1) the primary pointer is the mouse, in case of 2) the primary pointer is the touchscreen.

So, I'd say this is not an issue at all.

A question arose in my mind is what the fine pointer on the device is actually? There are four type of pointers we consider it as fine on Android, mouse, stylus, touchpad and trackball. Do OnePlus 3T and Moto G5 have stylus? It seems they have it? I am wondering whether we can tell user is using stylus or not..

Moving into GeckoView, this is Android specific issue.

Component: CSS Parsing and Computation → General
Product: Core → GeckoView
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: @media (pointer) and (hover) work incorrectly on OnePlus 3T → @media (pointer) reports `fine` as the primary pointer on OnePlus 3T and Moto G5

Hi. Thanks for the interest. Since none of you can test on an OnePlus 3T, I'm sharing some more info I've collected thanks to your insights.

The best guess I can tell is that such devices have multiple pointer devices and report a fine-grained type device prior to the coarse one.
what is the result on this test page, https://hiikezoe.github.io/interaction-media-features.html?

I think that's true also on OnePlus 3T. On Firefox 67 and Nightly 68.0a1 I get:

pointer feature
pointer: fine

any-pointer feature
any-pointer: fine
any-pointer: coarse

hover feature
hover: hover

any-hover feature
any-hover: hover

On Focus 8.0.9 and Chrome (default browser) I get:

pointer feature
pointer: coarse

any-pointer feature
any-pointer: fine
any-pointer: coarse

hover feature
hover: hover

any-hover feature
any-hover: hover

A question arose in my mind is what the fine pointer on the device is actually? There are four type of pointers we consider it as fine on Android, mouse, stylus, touchpad and trackball. Do OnePlus 3T and Moto G5 have stylus? It seems they have it?

OnePlus 3T does not have stylus or any of those fine pointers, neither "by design", nor that I have ever attached them (or at least since last reboot, I may have attached a mouse year ago).

On Chrome if there are both of fine and coarse type pointers on the device, they always report the coarse is the primary pointer, whereas we report the first pointer which founded by an Android API.

I think I can confirm the "crucial point" is here. Thanks to your links to Java code, I run a test on my OnePlus 3T, and here are all the pointers I found, sorted as the system returns them, with all their "source types":

-1: Virtual
Is pointer? NO
Is virtual? YES
 - SOURCE_CLASS_BUTTON
 - SOURCE_CLASS_NONE
 - SOURCE_DPAD
 - SOURCE_KEYBOARD
 - SOURCE_UNKNOWN

1: msm8996-tasha-mtp-snd-card Button Jack
Is pointer? NO
Is virtual? NO
 - SOURCE_CLASS_BUTTON
 - SOURCE_CLASS_NONE
 - SOURCE_KEYBOARD
 - SOURCE_UNKNOWN

2: msm8996-tasha-mtp-snd-card Headset Jack
Is pointer? NO
Is virtual? NO
 - SOURCE_CLASS_NONE
 - SOURCE_UNKNOWN

3: fpc1020
Is pointer? NO
Is virtual? NO
 - SOURCE_CLASS_BUTTON
 - SOURCE_CLASS_NONE
 - SOURCE_KEYBOARD
 - SOURCE_UNKNOWN

4: hbtp_vm
Is pointer? YES
Is virtual? NO
 - SOURCE_CLASS_NONE
 - SOURCE_CLASS_POINTER
 - SOURCE_MOUSE
 - SOURCE_UNKNOWN

5: qpnp_pon
Is pointer? NO
Is virtual? NO
 - SOURCE_CLASS_BUTTON
 - SOURCE_CLASS_NONE
 - SOURCE_KEYBOARD
 - SOURCE_UNKNOWN

6: gpio-keys
Is pointer? NO
Is virtual? NO
 - SOURCE_CLASS_BUTTON
 - SOURCE_CLASS_NONE
 - SOURCE_KEYBOARD
 - SOURCE_UNKNOWN

7: HWK,synaptics,s1302
Is pointer? NO
Is virtual? NO
 - SOURCE_CLASS_BUTTON
 - SOURCE_CLASS_NONE
 - SOURCE_KEYBOARD
 - SOURCE_UNKNOWN

8: synaptics,s3320
Is pointer? YES
Is virtual? NO
 - SOURCE_CLASS_BUTTON
 - SOURCE_CLASS_NONE
 - SOURCE_CLASS_POINTER
 - SOURCE_KEYBOARD
 - SOURCE_TOUCHSCREEN
 - SOURCE_UNKNOWN

The "is pointer" property is obtained using code from Gecko's InputDeviceUtils.isPointerTypeDevice(): if I understood it right, non-pointers are ignored when looking at devices. As you can see, there's a pointer named 4: hbtp_vm which is classified as a mouse, and it's reported before 8: synaptics,s3320 which should be the touch screen device.

I don't know what 4: hbtp_vm could be, unfortunately is not reported as virtual or as anything else that can be used to filter it out. From a quick Google search, it's something like a "virtual mouse", I don't know...

Here's the Java code I used to produce that list:

for (final int id : InputDevice.getDeviceIds()) {
	
	final InputDevice device = InputDevice.getDevice(id);
	Log.i("InputDevice", device.getId() + ": " + device.getName());
	
	final boolean isPointer = (device.getSources() & (
		InputDevice.SOURCE_CLASS_JOYSTICK |
		InputDevice.SOURCE_CLASS_POINTER |
		InputDevice.SOURCE_CLASS_POSITION |
		InputDevice.SOURCE_CLASS_TRACKBALL)) != 0;
	Log.i("InputDevice", "Is pointer? " + (isPointer ? "YES" : "NO"));
	Log.i("InputDevice", "Is virtual? " + (device.isVirtual() ? "YES" : "NO"));
	
	for (final Field f : InputDevice.class.getFields()) {
		if (f.getName().startsWith("SOURCE_")) {
			try {
				if (device.supportsSource(f.getInt(InputDevice.class))) {
					Log.v("InputDevice", " - " + f.getName());
				}
			} catch (IllegalAccessException e) {
				throw new RuntimeException(e);
			}
		}
	}
	
}

(In reply to Lorenzo Stanco from comment #13)

I don't know what 4: hbtp_vm could be, unfortunately is not reported as virtual or as anything else that can be used to filter it out. From a quick Google search, it's something like a "virtual mouse", I don't know...

Yeah, looks like it. I think you can complain it to the manufacturer. :)

(In reply to Hiroyuki Ikezoe (:hiro) from comment #14)

I'll search if they have a more technical reporting platform that is not their super crowded user forum. However, isn't it fine for Gecko to filter out that device by its exact name, once we're sure it can't represent a real physical mouse? I'm going to test if attaching a real mouse makes a new input device pop out, or if it somehow "uses" that virtual one.

I'm also curios to know Moto G5's input devices list, maybe Chris is experiencing the same "issue" because of the same exact cause.

(In reply to Lorenzo Stanco from comment #15)

(In reply to Hiroyuki Ikezoe (:hiro) from comment #14)

I'll search if they have a more technical reporting platform that is not their super crowded user forum. However, isn't it fine for Gecko to filter out that device by its exact name, once we're sure it can't represent a real physical mouse? I'm going to test if attaching a real mouse makes a new input device pop out, or if it somehow "uses" that virtual one.

I am reluctant to add such filtering without knowing why the manufacturer exposes the virtual mouse, even if the manufacturer accidentally exposed it, I am still reluctant because other manufacturer might have decent reasons.

FWIW, user can add an integer preference to forcibly set the primary pointer, the preference name is ui.primaryPointerCapabilities. If the value is 1, it means the device has a coarse type pointer. See here for other values.

Ok, I understand. I'll try to investigate it on OnePlus point of view, hope they give us a reply. Still curious about Moto G5, tho. Thanks for the tip about the preference!

Can we check whether a pointer device is actually enabled before including it in the list of pointer types? Or just exclude virtual devices? Android's InputDevice.isVirtual() documentation says: "Virtual input devices are provided to implement system-level functionality and should not be seen or configured by users." I'm curious what logic Chrome uses to choose coarse instead of fine.

https://developer.android.com/reference/android/view/InputDevice.html#isEnabled()
https://developer.android.com/reference/android/view/InputDevice.html#isVirtual()

(In reply to Chris Peterson [:cpeterson] from comment #18)

Can we check whether a pointer device is actually enabled before including it in the list of pointer types? Or just exclude virtual devices? Android's InputDevice.isVirtual() documentation says: "Virtual input devices are provided to implement system-level functionality and should not be seen or configured by users." I'm curious what logic Chrome uses to choose coarse instead of fine.

https://developer.android.com/reference/android/view/InputDevice.html#isEnabled()
https://developer.android.com/reference/android/view/InputDevice.html#isVirtual()

Nice! Checking isEnabled sounds promising. Lorenzo, could you please check the result of the function on your device?

As for isVirtual(), even if it can't be used for Lorenzo's case, it might work on Moto G5. I will provide a try build with those changes.

Thank you, Chris!

Flags: needinfo?(lorenzo.stanco)

Checking isEnabled sounds promising. Lorenzo, could you please check the result of the function on your device?

Unfortunately isEnabled() is available only on API 27+, i.e. Android 8.1, and I'm on Android 8.0. I should be able to upgrade to Android 9 soon, they announced the release just a couple weeks ago. As soon as the OTA arrives, I'll try it.

Flags: needinfo?(lorenzo.stanco)
Flags: needinfo?(lorenzo.stanco)

(In reply to Hiroyuki Ikezoe (:hiro) from comment #20)

Just pushed a try;
https://treeherder.mozilla.org/#/jobs?repo=try&revision=de8c832a07c8d897435e8db698b6aabdab3e7ddd&selectedJob=250702350

I'd like to know the result on https://hiikezoe.github.io/interaction-media-features.html .

I tested your Try build on my Moto G5 and the results are the same as Fennec 67:

pointer feature
pointer: fine

any-pointer feature
any-pointer: fine
any-pointer: coarse

hover feature
hover: hover

any-hover feature
any-hover: hover
Flags: needinfo?(cpeterson)
status-firefox67: affected → wontfix
status-firefox67.0.1: --- → wontfix
Priority: -- → P3
Whiteboard: [geckoview]

Checking isEnabled sounds promising. Lorenzo, could you please check the result of the function on your device?

Unfortunately isEnabled() is available only on API 27+, i.e. Android 8.1, and I'm on Android 8.0. I should be able to upgrade to Android 9 soon.

Unfortunately on Android 9 the hbtp_vm is still there, and the new API isEnabled() returns true for it :( Still no reply from OnePlus, as expected... I'm out of ideas.

Flags: needinfo?(lorenzo.stanco)

(In reply to Makoto Kato [:m_kato] from comment #24)

Also, when I check Chromium code, they return coarse (https://source.chromium.org/chromium/chromium/src/+/master:ui/base/pointer/pointer_device_android.cc;l=41;drc=8586102b48a409c634250f31c3fc7bdde5ec3db0) if one device has touch at least.

That's what I referred in comment 11.

See Also: → 1558865

Since I receive same issue multiple times in http://github.com/mozilla-mobile/fenix, I will take this.

  • Chrome returns coarse if something input device has coarse capability.
  • Firefox for Windows seems to return coarse if it is tablet mode.
  • We cannot assume what is primary input device from Android API. "Primary" word is too vague as Android API.

So we should return coarse like Chrome if any input device has coarse capability. Most web developers checks the behavior on Chrome, so it will still be web compatibility.

Assignee: nobody → m_kato

Many users report that GeckoView returns non-coarse value for primary pointer
capabilities. Actually, we assume that primary pointer device is 1st device by
InputDevice.getDeviceIds, but some mobile devices seems not to return touch
screen according to user reports.

Also, Android's InputDevice API doesn't define what is primary pointer
device, so we cannot assume primary pointer device well.

Other implmenents are,

  • Chrome returns coarse if something input device has coarse capability.
  • Firefox for Windows seems to return coarse if it is tablet mode.

So we should return coarse as primary pointer capability when having touch
screen, like Chrome.

Pushed by m_kato@ga2.so-net.ne.jp: https://hg.mozilla.org/integration/autoland/rev/59184575dca0 Primary pointer capability should be coarse when having touch screen. r=geckoview-reviewers,agi

https://hg.mozilla.org/mozilla-central/rev/59184575dca0

Status: NEW → RESOLVED
Closed: 3 years ago
status-firefox96: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 96 Branch

Thanks!

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: