1918292 - Windows touchscreen devices report as "not a tablet" if they lack an auto-rotation sensor

Status: UNCONFIRMED

(Core :: Widget: Win32, defect, P3)

Description

[Michael]

Steps to reproduce:

You'll need a Windows 11 PC with a touchscreen attached, no keyboard or mouse devices attached (even virtual ones – this can be hard!), and no built-in rotation sensor. I'm using a desktop PC with a HDMI monitor that has a built-in multi-touch USB HID touch panel.

1. Visit https://mustaqahmed.github.io/web/mq-test.html (site not operated by me, but shows the result of pointer, hover, any-pointer and any-hover media queries).

2. Observe pointer, hover, any-pointer and any-hover results.

Actual results:

Firefox currently reports:

• pointer: fine
• hover: hover
• any-pointer: coarse + fine
• any-hover: hover

Expected results:

Instead, this should report:

• pointer: coarse
• hover: none
• any-pointer: coarse
• any-hover: none

Comment 1

[Michael]

Related links:

• Related bug, about Firefox conflating "is it a tablet" with using a touch UI: https://bugzilla.mozilla.org/show_bug.cgi?id=1735765
• Same bug, but in Chrome/Chromium: https://issues.chromium.org/issues/366055333

It appears Firefox treats the absence of an auto-rotation sensor as a reason to say this is not a tablet:

  // If the device is not supporting rotation, it's unlikely to be a tablet,
  // a convertible or a detachable. See:
  // https://msdn.microsoft.com/en-us/library/windows/desktop/dn629263(v=vs.85).aspx

Chrome includes the same comment with nearly-identical wording.

However:

• Nothing on the referenced MSDN doc for AR_STATE says anything about "detecting a tablet"

• "Whether something is a tablet" is not a correct way to classify how Firefox should report pointer, hover, any-pointer and any-hover.

While it's true the device I'm using is not a tablet (it's a desktop PC with a HDMI monitor that has a built-in multi-touch USB HID sensor), that shouldn't affect the result of the pointer, hover, any-pointer and any-hover queries.

This is a usability problem for websites – as they cannot present a touch-friendly UI using media queries.

Possibly relevant about:support flags:

• Pointing devices: Touchscreen and mouse
• ui.osk.debug.keyboardDisplayReason: IKPOS: Lack of keyboard confirmed.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[Michael]

A couple of notes following feedback on the Chrome bug, for if you're attempting to reproduce this issue:

• It is not enough to use a device with an auto-rotation where when sensor is disabled, because then GetAutoRotationState will return AR_DISABLED, which Chrome and Firefox are happy to treat as a "tablet".

• Chrome's chrome://system page has a usb_keyboard_attached debug string, which could be helpful – as it follows similar logic. You want a device where that says:

  Keyboard Detected:
  Not a tablet deviceAR_NOSENSOR
  

  And nothing more.

To explain why I think media queries are connected to tablet mode:

• Gecko_MediaFeatures_PrimaryPointerCapabilities requests LookAndFeel::IntID::PrimaryPointerCapabilities
• The request is evaluated in nsLookAndFeel::NativeGetInt, which calls WinUtils::GetPrimaryPointerCapabilities().

WinUtils::GetPrimaryPointerCapabilities does these steps:

• if IsTabletDevice() returns true, it returns PointerCapabilities::Coarse
  • My Windows PC with a touch screen fails this check, because it doesn't have an auto-rotation sensor (GetAutoRotationState() == AR_NOSENSOR)
• if SystemHasMouse() returns true, it returns PointerCapabilities::Fine | PointerCapabilities::Hover.
  • That check returns GetSystemMetrics(SM_MOUSEPRESENT).
  • The MSDN docs note: "Nonzero if a mouse is installed; otherwise, 0. This value is rarely zero, because of support for virtual mice and because some systems detect the presence of the port instead of the presence of a mouse. "
  • When a touchscreen and no mouse is connected to my Windows computer:
    • GetSystemMetrics(SM_MOUSEPRESENT) returns 1.
  • When I unplug the touchscreen and only use a keyboard on my Windows computer:
    • GetSystemMetrics(SM_MOUSEPRESENT) returns 0.
    • The media queries pointer, hover, any-pointer and any-hover all report none.
• if WinUtils::IsTouchDeviceSupportPresent() returns true, it returns PointerCapabilities::Coarse.
  • That in turn checks GetSystemMetrics(SM_DIGITIZER) has NID_READY and either NID_INTEGRATED_TOUCH or NID_EXTERNAL_TOUCH.
  • When my touchscreen is plugged in, that returns 0xC1 (NID_READY | NID_MULTI_INPUT | NID_INTEGRATED_TOUCH)
  • That would return true on my Windows computer, but the check is never evaluated because SystemHasMouse() returns true.
• Otherwise, it returns PointerCapabilities::None.

I got the values of GetSystemMetrics and GetAutoRotationState with a little Python code, which works on the version of Python on the Windows Store:

import ctypes
ctypes.windll.user32.GetSystemMetrics(19) # SM_MOUSEPRESENT
hex(ctypes.windll.user32.GetSystemMetrics(94)) # SM_DIGITIZER
ret = ctypes.c_int32()
ctypes.windll.user32.GetAutoRotationState(ctypes.by_ref(ret))
hex(ret.value)

Comment 4

[Ray Kraesig [:rkraesig]]

I'm going to somewhat tentatively mark this as dependent on bug 1735765. (Newcomers to this bug should note that further detailed discussion is taking place there.)