Closed Bug 1638556 Opened 5 years ago Closed 1 years ago

incorrect CSS media query pointer: coarse (not fine) on Windows 10 desktop/laptop with trackpad/touchpad and no external mouse

Categories

(Core :: Widget: Win32, defect, P2)

Product:
Core
Component:
Widget: Win32
Version:
76 Branch
Type:
defect

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: lsemprini, Unassigned)

References

Details

Attachments

(4 files)

mozilla-pointer-matchMedia-incorrect-win10-desktop.png
34.76 KB, image/png
Details
chrome-pointer-matchMedia-correct-win10-desktop.png
41.47 KB, image/png
Details
win10-system-props.png
48.27 KB, image/png
Details
mozilla-pointer-matchMedia-corrected-in-v82-for-win10-desktop.png
33.74 KB, image/png
Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:76.0) Gecko/20100101 Firefox/76.0

Steps to reproduce:

On a Windows 10 laptop with a trackpad but not an external mouse, Firefox gives INCORRECT false results for:

window.matchMedia("(pointer: fine").matches -> false - WRONG
window.matchMedia("(any-pointer: fine").matches -> false - WRONG

You can easily check/reproduce this on this handy page:

https://patrickhlauke.github.io/touch/pointer-hover-any-pointer-any-hover/

Notice that as soon as you plug in an external mouse (even without reloading the page), the incorrect "coarse" values become "fine." But with only a trackpad, the values incorrectly show as "coarse."

This is in direct violation of the CSS Media Queries spec, section 7, "Interaction Media Features" which states unequivocally:

https://www.w3.org/TR/mediaqueries-4/#mf-interaction
"Typical examples of devices matching combinations of pointer and hover:
"pointer: fine" mouse, touch pad, advanced stylus digitizers (Surface, Samsung Note, Wacom Intuos Pro, etc) "

Notice how mouse and trackpad/touchpad both MUST return "fine," not "coarse."

Chrome returns "fine" as expected.

I tracked this bug down to the exact line of code, which was added on 4 Oct 2018 and released on Firefox/Mozilla 64 as part of a shaky Windows Tablet workaround (which was not intended to affect desktop, but did).

The bug "fixed" was:
https://bugzilla.mozilla.org/show_bug.cgi?id=1493128
and
https://bugzilla.mozilla.org/show_bug.cgi?id=1495938

And the incorrect code is in widget/windows/WinUtils.cpp:

https://hg.mozilla.org/mozilla-central/rev/bf098432003adaa3862f8271de486ee3bdbd8362

static bool
IsMousePresent()
{

  • return ::GetSystemMetrics(SM_MOUSEPRESENT);
  • if (!::GetSystemMetrics(SM_MOUSEPRESENT)) {
  • return false;
  • }
  • DWORD count = InputDeviceUtils::CountMouseDevices();
  • if (!count) {
  • return false;
  • }
  • // If there is a mouse device and if this machine is a tablet or has a
  • // digitizer, that's counted as the mouse device.
  • // FIXME: Bug 1495938: We should drop this heuristic way once we find out a
  • // reliable way to tell there is no mouse or not.
  • if (count == 1 &&
  •  (WinUtils::IsTouchDeviceSupportPresent() ||

  •   IsTabletDevice())) {
  • return false;
  • }

note SM_MOUSEPRESENT is NOT the problem: it always returns 1 on laptops with trackpads.

It is the last piece of code above that returns false when there is only a trackpad present, and this causes the calling code to return Coarse instead of Fine:

if (IsMousePresent()) {

  • return PointerCapabilities::Fine|
  • return PointerCapabilities::Fine |
    PointerCapabilities::Hover;
    }

if (IsTouchDeviceSupportPresent()) {
return PointerCapabilities::Coarse;
}

Note that WinUtils::IsTouchDeviceSupportPresent() is:

WinUtils::IsTouchDeviceSupportPresent()
{
int32_t touchCapabilities = ::GetSystemMetrics(SM_DIGITIZER);
return (touchCapabilities & NID_READY) &&
(touchCapabilities & (NID_EXTERNAL_TOUCH | NID_INTEGRATED_TOUCH));
}

and I verified this is TRUE on a laptop (HP Omen Windows 10) with a trackpad.

So this is not correct. Need to find a different workaround for the tablet problem.

This code is being incorrectly applied on non-Tablet devices.

It happens that my laptop does not have a touchscreen, but that is not really relevant. On any Windows 10 laptop with a trackpad/touchpad, these media queries should return "fine" as per the spec.

Actual results:

When external mouse is not plugged in but trackpad/touchpad exists:
window.matchMedia("(pointer: fine").matches -> false - WRONG
window.matchMedia("(any-pointer: fine").matches -> false - WRONG

When external mouse is plugged in and trackpad/touchpad exists:
window.matchMedia("(pointer: fine").matches -> true - RIGHT
window.matchMedia("(any-pointer: fine").matches -> false - RIGHT

Expected results:

Regardless of whether external mouse is plugged in or not, when touchpad/trackpad exists:

window.matchMedia("(pointer: fine").matches -> true - RIGHT
window.matchMedia("(any-pointer: fine").matches -> true - RIGHT

Oh, and just to be clear, the spec behavior is definitely better because people use this CSS media query to decide whether to present big finger-sized UI elements or tiny mouse/trackpad-sized UI elements.

Ah, and also in case anyone wonders, my Windows 10 laptop is NOT in "tablet mode" (which can be toggled using a tile in the weird Windows 10 "action center" menu that comes up when you click the icon at the lower right of the screen). And as I mentioned above, my laptop is not a touchscreen device, but that's not actually relevant (even if it were, the results for "pointer" and "any-pointer" should be "fine" whenever a touchpad/trackpad is present).

Louis, thanks for this bug report.

I can understand your irritation. Do you have tablet devices and do you mind sharing the results on the devices? I have no tablet devices so that I can't check it unfortunately. IIRC, with the way you suggested the result was not what we expected at all. As far as I know Windows didn't provide any good way to detect/distinguish mouse/touch pad devices there. Things may have changed since then.

Hi Hiroyuki!

Unfortunately no I don't have any Windows tablet devices.

I'm not even 100% sure what type of device your original change was meant for, perhaps the Surface devices? If we can identify the type of Windows devices for which we need the heuristic, perhaps we can add an additional test so that regular old laptops that have trackpads/touchpads are not affected by the heuristic.

It seems likely that something in Windows changed between 2018 and now.

My best guess is that the behavior of ::GetSystemMetrics(SM_DIGITIZER) changed in some Windows update.

I tested whether perhaps the Windows code called by InputDeviceUtils::CountMouseDevices() changed behavior in some Windows update. That code is:

+DWORD
+InputDeviceUtils::CountMouseDevices()
+{
+  HDEVINFO hdev = SetupDiGetClassDevs(&GUID_DEVINTERFACE_MOUSE,
+                                      nullptr,
+                                      nullptr,
+                                      DIGCF_DEVICEINTERFACE | DIGCF_PRESENT);
+  if (hdev == INVALID_HANDLE_VALUE) {
+    return 0;
+  }
+
+  DWORD count = 0;
+  SP_INTERFACE_DEVICE_DATA info = {};
+  info.cbSize = sizeof(SP_INTERFACE_DEVICE_DATA);
+  while (SetupDiEnumDeviceInterfaces(hdev,
+                                     nullptr,
+                                     &GUID_DEVINTERFACE_MOUSE,
+                                     count,
+                                     &info)) {
+    if (info.Flags & SPINT_ACTIVE) {
+      count++;
+    }
+  }
+  SetupDiDestroyDeviceInfoList(hdev);
+  return count;
+}

I thought perhaps this code now says that there are 0 mice in Windows when only a trackpad is present, whereas it used to say there is 1 mouse when only a trackpad is present. If that were the case, then code that causes the incorrect "coarse" value would actually be just above the heuristic:

DWORD count = InputDeviceUtils::CountMouseDevices();
if (!count) {
return false;
}

HOWEVER it is not the case. I wrote a small C++ program with the contents of InputDeviceUtils::CountMouseDevices() and when I have only a trackpad, Windows reports 1 device.

Here is the code in case it is of use to anyone:

#include <stdlib.h>
#include <stdio.h>
#include <type_traits>
#include <functional>
#include <assert.h>
#include <windows.h>

#include <dbt.h>
#include <hidclass.h>
#include <ntddmou.h>
#include <setupapi.h>
#include <Ntddmou.h>

const GUID LOCAL__GUID_DEVINTERFACE_MOUSE = 
{0x378DE44C, 0x56EF, 0x11D1, 0xBC, 0x8C, 0x00, 0xA0, 0xC9, 0x14, 0x05, 0xDD};
// from ms dox:
// https://docs.microsoft.com/vi-vn/windows-hardware/drivers/install/guid-devinterface-mouse
/*
{  378DE44C   -56EF-   11D1-  BC     8C-   00    A0    C9    14    05    DD}
*/

void doit()
{
    printf("mouse=%lx\n", GetSystemMetrics(SM_MOUSEPRESENT));
    printf("dig=%lx\n", GetSystemMetrics(SM_DIGITIZER));
    printf("dig_ready=%lx\n", GetSystemMetrics(SM_DIGITIZER)&NID_READY);
    printf("dig_ext=%lx\n", GetSystemMetrics(SM_DIGITIZER)&NID_EXTERNAL_TOUCH);
    printf("dig_int=%lx\n", GetSystemMetrics(SM_DIGITIZER)&NID_INTEGRATED_TOUCH);

    HDEVINFO hdev = SetupDiGetClassDevs(&LOCAL__GUID_DEVINTERFACE_MOUSE,
                                        nullptr,
                                        nullptr,
                                        DIGCF_DEVICEINTERFACE | DIGCF_PRESENT);
    if (hdev == INVALID_HANDLE_VALUE) {
        assert(0);
    }
    
    DWORD count = 0;
    SP_INTERFACE_DEVICE_DATA info = {};
    info.cbSize = sizeof(SP_INTERFACE_DEVICE_DATA);
    while (SetupDiEnumDeviceInterfaces(hdev,
                                       nullptr,
                                       &LOCAL__GUID_DEVINTERFACE_MOUSE,
                                       count,
                                       &info)) 
    {
        if (info.Flags & SPINT_ACTIVE) {
            count++;
        }
    }
    SetupDiDestroyDeviceInfoList(hdev);
    printf("count is %d", count);
}

int main()
{
    doit();
}

BTW that test code needs to be linked with SetupApi.lib to compile.

Thanks for testing. Yes, we'd like to know a way to tell the difference between Surface's touch screen (and other touch screens) and trackpad/touchpad. As you pointed out, IsTouchDeviceSupportPresent seems to be an overkill for some devices.

There are a lot of things you could try.

What is it exactly that makes Surfaces (and the other devices you want to capture) different from regular laptops? Do they run a different kind/verison of Windows and can you detect that?

Remember that we don't need/want to detect laptops that have touchscreens, because those laptops are also going to have trackpads and so we want the "fine" return value. We ONLY want to detect Surface-like devices that have ONLY a touchscreen and not necessarily a mouse and no trackpad/touchpad.

Dox for GetSystemMetrics(SM_DIGITIZER) explain the flags. Perhaps one of them varies based on touchpad/trackpad vs. touch screen?

https://docs.microsoft.com/en-us/windows/win32/wintouch/getting-started-with-multi-touch-messages
TABLET_CONFIG_NONE 	0x00000000 	The input digitizer does not have touch capabilities.
NID_INTEGRATED_TOUCH 	0x00000001 	An integrated touch digitizer is used for input.
NID_EXTERNAL_TOUCH 	0x00000002 	An external touch digitizer is used for input.
NID_INTEGRATED_PEN 	0x00000004 	An integrated pen digitizer is used for input.
NID_EXTERNAL_PEN 	0x00000008 	An external pen digitizer is used for input.
NID_MULTI_INPUT 	0x00000040 	An input digitizer with support for multiple inputs is used for input.
NID_READY 	0x00000080 	The input digitizer is ready for input. If this value is unset, it may mean that the tablet service is stopped, the digitizer is not supported, or digitizer drivers have not been installed.
Checking the NID_* values is a useful way of checking the capabilities of a user's computer to configure your application for touch, pen, or non-tablet input. For example, if you have a dynamic user interface (UI) and want to automatically configure some of it, you could check for NID_INTEGRATED_TOUCH, NID_MULTITOUCH, and could get the maximum number of touches the first time that a user runs your application.

On my laptop the return value is 0xc1 so NID_INTEGRATED_TOUCH|NID_MULTI_INPUT|NID_READY

Another idea is maybe GetSystemMetrics(LOCAL_MAXTOUCHES_INDEX)

    const int LOCAL_MAXTOUCHES_INDEX = 95;
    printf("mt=%lx\n", GetSystemMetrics(LOCAL_MAXTOUCHES_INDEX));

...but on my laptop (which does NOT have a touchscreen) it returns 2 (since I guess my touchscreen can do limited multi-touch gestures).

TouchCapabilities.TouchPresent seems to be a likely answer but not sure how to access it from a Win32 program:

https://stackoverflow.com/questions/13004911/how-to-specifically-detect-a-touch-screen-display-on-windows-8
https://docs.microsoft.com/en-us/uwp/api/Windows.Devices.Input.TouchCapabilities?redirectedfrom=MSDN&view=winrt-19041#Windows_Devices_Input_TouchCapabilities_TouchPresent

...but on my laptop (which does NOT have a touchscreen) it returns 2 (since I guess my touchscreen can do limited multi-touch gestures).

sorry, that should say my touchpad/trackpad can do limited multi-touch gestures.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: Untriaged → Widget: Win32
Product: Firefox → Core

The severity field is not set for this bug.
:jimm, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(jmathies)
Severity: -- → S3
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(jmathies)
Keywords: good-first-bug
Priority: -- → P2
Attached image win10-system-props.png —

Just wanted to add that same issue is found on Win10 desktop machines. Chrome (v 86.04240.75) and Edge (obviously) report correct results using test page (https://patrickhlauke.github.io/touch/pointer-hover-any-pointer-any-hover/) -- but Firefox (v 81.0.2) does not. See screenshots attached. Also note Win10 System Properties screen -- perhaps that is part of the problem (but only for FF.)

Happy to report that Fireforx (v 82.0) update resolves the issue for Win10 desktop. New test = only diff between Chrome and FF now is how any-pointer: coarse returns false for FF (correct). Time to file bug with Chromium about their incorrect results for same.

The issue persists in FF 82.0.2 (64bit) on Windows 10. I'll keep an eye on the progress, if you need any further info, just ask!

See Also: → 1747633

I'm going to remove the good-first-bug tag from this bug since it requires testing across a variety of hardware, which new contributors don't necessarily have access to.

Keywords: good-first-bug

This general issue of wrong input media query values seems to also currently apply to desktop PCs that have graphics tablets attached (even if there is a mouse also present). Unplugging my tablet causes the test above to display the correct values, fixing some sites, and plugging it back in causes it to display incorrect values again, breaking some sites. Annoyingly, the main site that's broken by this for me is pixiv, an art site, and artists are more likely to have graphics tablets. (Various navigation arrows on pixiv disappear if the browser is in "coarse pointer, no hover" mode.)

A desktop PC with a graphics tablet attached should not be treated the same way as a tablet PC that has no mouse. In particular, graphics tablets are "precise" and do let the user "hover" the cursor over things, unlike touch screens. In the possible case that there's no way for the browser to know whether a tablet input device is a graphics tablet or a touch screen, it should base the media queries based on recent input event data rather than hardware presence. If there's no way to detect what kind of hardware the user is using based on input events, or the hardware lies about its capabilities, then there needs to be a user-facing toggle or override.

Using 103.b3 (developer edition) on Windows 10. "Tablet mode" is not enabled in Windows, but Windows Ink is enabled in my tablet software's settings.

On Firefox 104.0.2, both @media (hover: hover) and @media (pointer: fine) do not work for me on a Windows 10, a graphic tablet driver installed but not currently plugged.

OS:
Edition Windows 10 Pro
Version 22H2
Installiert am ‎10.‎06.‎2020
Betriebssystembuild 19045.2130
Leistung Windows Feature Experience Pack 120.2212.4180.0

Firefox-Build: 106.0.3 (64-Bit)

Input:
-Touchpad
-Mouse
-Touchscreen

Results:
pointer:none false
pointer:coarse true
pointer:fine false
hover:none true
hover:hover false
any-pointer:none false
any-pointer:coarse true
any-pointer:fine true
any-hover:none false
any-hover:hover true

See Also: → 1806259
See Also: → 1851244

Hi all,

Good news! The any-pointer and any-hover portions of this bug were fixed in Bug 1813979.

The pointer and hover are a bit more complex, but I will be fixing them as part of Bug 1851244.

Status: NEW → RESOLVED
Closed: 1 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: