CSS media queries wrongly detect a Win10 desktop computer with a mouse and a touchscreen, as a device with no mouse (hover: none) and a touchscreen (pointer: coarse)
Categories
(Core :: Widget: Win32, defect, P3)
Tracking
()
People
(Reporter: cob.bzmoz, Unassigned)
References
Details
(Whiteboard: [win:touch])
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:107.0) Gecko/20100101 Firefox/107.0
Steps to reproduce:
Go to this page with Firefox stable version 107.0.1 or with Firefox Nightly 110.0a1 (2022-12-17) (64-bit), on a Windows 10 PC with a vertical mouse connected and no touch devices connected:
https://patrickhlauke.github.io/touch/pointer-hover-any-pointer-any-hover/
Firefox stable user agent was: "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:107.0) Gecko/20100101 Firefox/107.0"
Firefox Nightly user agent was: "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:110.0) Gecko/20100101 Firefox/110.0"
Then, visit the same page with Microsoft Edge (i.e. Chromium).
I have additional information to provide, but this form won't let me. I'll post a comment.
Actual results:
In both Firefoxes:
pointer:none is false
pointer:coarse is true
pointer:fine is false
hover:none is true
hover:hover is false
any-pointer:none is false
any-pointer:coarse is true
any-pointer:fine is false
any-hover:none is true
any-hover:hover is false
Expected results:
Results should have been consistent with what I saw in Edge, which was:
pointer:none is false
pointer:coarse is false
pointer:fine is true
hover:none is false
hover:hover is true
any-pointer:none is false
any-pointer:coarse is true
any-pointer:fine is true
any-hover:none is false
any-hover:hover is true
So I've spent the last few hours tracking this down, digging through Firefox bug reports and source code, and even some Chromium source code. I even wrote a console application so I could see some of the same Win32 stuff Firefox and Chrome query. Here's what I've found.
Firefox will regard your computer as having touch support if GetSystemMetrics(SM_DIGITIZER) has flag NID_READY and any of flags NID_EXTERNAL_TOUCH | NID_INTEGRATED_TOUCH set.
Even if GetSystemMetrics(SM_MOUSEPRESENT) is truthy, Firefox will regard your computer as not having a mouse if your computer has touch support and only one mouse.
The check for whether your device is a tablet can be found here.
Currently, my computer has a vertical mouse and no touch devices of any kind plugged in. Chromium handles media queries properly; Firefox mishandles them as described above. I wrote code to dump a ton of the Win32 state that Firefox is interested in...
GetSystemMetrics(SM_DIGITIZER)is 0xC1 (integrated touch; ready).GetSystemMetrics(SM_MAXIMUMTOUCHES)is 0x100.GetSystemMetrics(SM_MOUSEPRESENT)is 1.GetSystemMetrics(SM_SYSTEMDOCKED)is 0.GetSystemMetrics(SM_TABLETPC)is 1.
Using GetPointerDevices, I see that I have exactly one digitizer. The productString is \??\VIRTUAL_DIGITIZER; the pointerDeviceType is POINTER_DEVICE_TYPE_TOUCH. I can't find any real information on this product string, but I did find that Google Chrome ended up having to fix a bug where it would randomly appear out of nowhere and be detected as an Xbox gamepad. Between that and the name, it clearly doesn't reflect any real hardware. I don't see anything in Device Manager's HID category that looks like "virtual digitizer."
Using GetAutoRotationState, the only flag set is AR_NOSENSOR, which disqualifies my computer from being recognized as a tablet per Firefox's logic.
Using SetupDiEnumDeviceInterfaces and friends, I can confirm that the mouse that Firefox sees (when it counts how many mice I have) is my vertical mouse. I can't figure out how to print the name, but the instance ID is the same as what I see in Device Manager. The only flag set is SPINT_ACTIVE.
So Firefox sees my actual mouse, and no other mouses, but also this weird out-of-nowhere "virtual digitizer" -- something that Chrome has already had their own struggles with, and needed to filter out. The combination of the two causes CSS media queries to completely malfunction; for Firefox only, they cannot reliably be used to identify touch-only devices.
So why isn't this broken in Chromium? Their checks for pointer type availability (none/coarse/fine) and hover availability are here. In both cases, they only assume that a device is touch-only if the device is a tablet. Their tablet check is similar to yours, and my computer is recognized as "not a tablet" for the same reason in Chromium (AR_NOSENSOR).
Chromium allows hover based solely on a non-tablet device having a truthy GetSystemMetrics(SM_MOUSEPRESENT), so my device is properly recognized as supporting hover. Chromium does mistakenly list my device as supporting a coarse pointer, but it also lists it as supporting a fine pointer.
There is some prior knowledge on this. I believe this may be bug 1747633, but that bug's submitter mistakenly marked it as RESOLVED INVALID and then never reopened it even after repro'ing it again. I can't reopen it myself.
This bug appears to be the direct result of workarounds taken to """fix""" bug 1493128, leading to bug 1495938 as a new issue. The latter was marked as RESOLVED WORKSFORME, so I can't reopen that either.
My blind random guess as to a fix would be: have Firefox ignore GetSystemMetrics(SM_DIGITIZER) if the only device returned by GetPointerDevices is the virtual digitizer. However, this problem happened as the result of a Firefox-side hack used to ensure that a Surface's touch screen would be detected. Is the Surface's digitizer the same virtual digitizer that my computer is pretending to have? The original bugs don't list this sort of information.
|
||
Updated•1 year ago
|
There's a new complication to this.
My PC forcibly restarted due to a Windows update. When it did, I booted up Firefox and found that it was properly handling media queries. So the problem fixed itself, right? Well, not quite.
I booted up my custom console application for reporting all of the Win32 values that Firefox uses here, and it was reporting the same values as lat night: digitizer values set; "virtual digitizer" present. That seemed odd. Did I miss something when going through Firefox's source code? I spent maybe fifteen or twenty minutes rechecking. I did see that the code reports the platform-default input device (based on how Firefox was compiled) if resist fingerprinting is on; I had resist fingerprinting enabled on stable, so I turned that off, and saw no change. (And now I'm wondering how the hell the media queries even broke previously.)
Then it occurred to me to fire up Nightly and see what that did. And sure enough: Nightly was reporting the wrong values. So right now, as I write this, I have two different Firefoxes (108.0.1 64-bit release, and 110.0a1 (2022-12-17) 64-bit nightly) running, and one of them is reporting correct values and the other is not.
So I see three possibilities:
a) The "virtual digitizer" is not present immediately after system startup, and becomes available either only after a delay or only in response to some unknown event happening anywhere in the system. If Firefox only checks input-related CSS media queries one time (or if it only re-checks under some circumstance that I don't know how to trigger), then Firefox will (if started before the virtual digitizer shows up) never see the virtual digitizer and will therefore run properly until restarted.
b) Windows reports different GetSystemMetrics values to different processes, even if they are, fundamentally, the same program. This seems unlikely.
c) The underlying input checks work properly, but the media query check that wraps them does not. If, somehow, mozilla::dom::BrowsingContext::TouchEventsOverride got stuck for every browsing context, then that would cause the underlying input checks to be skipped and Firefox to return a coarse non-hovering pointer in all tabs.
After I send this, I'll restart the (stable) browser and see if it starts to break, as a test for a.
After restarting Firefox stable, it is continuing to show correct CSS media query values. I thought that that might rule out a. However, it doesn't... because I also ran my "report the raw Win32 values" program again, after testing Firefox stable, and I now see that GetSystemMetrics(SM_DIGITIZER) is 0.
So, the virtual digitizer can appear and disappear seemingly at will.
So I ran one final test -- and I checked SM_DIGITIZER after each step.
- Open example.com.
- Open the browser debug tools (F12) on example.com.
- Enable Responsive Design Mode on example.com.
- While in Responsive Design Mode, click and drag on the page to simulate touch input.
- Exit Responsive Design Mode.
- Close the browser debug tools on example.com.
- Refresh the example.com tab in an attempt to reset it.
- Close the example.com tab.
Step 4 triggered the virtual digitizer to appear. So based on my experience yesterday and this test now, I believe I've identified the problem: simulating any touch input on a page using Responsive Design Mode will spawn Windows's virtual digitizer, and the digitizer will not shut down again until Firefox is closed entirely. As long as the virtual digitizer is active, Firefox (all processes and installations) will mistake your computer for a touch-only device due to bug 1495938.
The good thing is that at the very least, this means that users who aren't web developers should never run into this. CSS media queries will be safe for most people. They won't, however, be safe for any of the people who specifically need to use and test CSS media queries.
To clarify my previous comment: the virtual digitizer remained present even after steps 5, 6, 7, and 8.
I've found two relevant pieces of code, but can't figure out how they connect.
Responsive Design Mode enables touch emulation by setting state on the browsing context(s?), but I can't figure out what that actually causes -- what happens under the hood.
However, on a hunch, I searched up the Windows APIs for touch emulation. It would seem that Firefox uses CreateSyntheticPointerDevice, but never uses DestroySyntheticPointerDevice later on. The caller, InitPenInjection, is called by nsWindow::SynthesizeNativePenInput. It seems like the end "goal" of this system is to allow the injection of natively-handled touch events from methods like these.
This has a few implications:
- Any program on the system that enables touch emulation will break CSS media queries in Firefox as long as touch emulation is enabled.
- Once Firefox itself enables touch emulation, it will remain enabled until the offending Firefox instance (all windows) is closed.
If Firefox were modified to destroy its synthetic pointer device when no longer in use, then that would at least allow Firefox to behave normally when RDM is turned off. However, turning it on in any tab would still break CSS media queries for every other tab (and across all running instances of Firefox, at that).
Because Firefox already fakes CSS media queries manually when RDM is open (it forces a coarse-only pointer in the target browsing context), Firefox could be made to ignore the virtual digitizer for CSS media queries and/or IsMousePresent generally without that then leading to RDM tabs testing (in CSS media queries) as having fine hovering pointers.
So, a summary:
Once you simulate any touch inputs in RDM, all CSS media queries for pointer, hover, any-pointer, and any-hover in all running instances of Firefox will break until the Firefox instances that you used RDM in is closed. Additionally, this will also happen while any program running on the system is using OS-level touch emulation.
Repro steps
- Go here in any other tab or window, not in RDM, to check the results of your CSS media queries prior to this bug occurring.
- Open Responsive Design Mode in any tab.
- Click and drag anywhere on the RDM-previewed page, to simulate touch events.
- Go here in any other tab or window, not in RDM, to check the results of your CSS media queries.
Cause
Multifaceted:
- Firefox creates an OS-level synthetic pointer device for RDM, but never destroys it. This causes Windows to create a virtual digitizer.
- If your system has only one mouse and tests as having a digitizer present (
SM_DIGITIZER), Firefox assumes that that mouse is the digitizer, for compatibility with Surface laptops.
Potential fix
If the Surface does not itself rely on the virtual digitizer for its touch screen, then the fix is simple. However, only someone with a Surface could check this.
Modify IsMousePresent's compatibility hack for the Surface. You should ignore a digitizer if its product string (POINTER_DEVICE_INFO::productString as retrieved by GetPointerDevices) is \??\VIRTUAL_DIGITIZER, or if its raw input device name (GetRawInputDeviceInfo using RIDI_DEVICENAME) is \\?\VIRTUAL_DIGITIZER (note the alternate punctuation). The respective string constants for copying and pasting into source code are "\\??\\VIRTUAL_DIGITIZER" and "\\\\?\\VIRTUAL_DIGITIZER. I think only one of those checks should be needed, so pick whichever best suites your test.
- best suites your tastes
I wish this site had an edit button.
|
||
Comment 9•1 year ago
|
||
cc :cmartin for your info.
|
||
Comment 10•9 months ago
|
||
Hi there,
Woah! Thank you for doing so much investigation into this. Your notes were very helpful!
First, I have some good news! The any-pointer and any-hover parts of this should've been fixed in Bug 1813979, and that was released in Firefox 117, so that should be all good for you now.
The pointer and hover media queries are still returning results that aren't great (because of the whole "if there's a touchscreen you need two mouses" check that you noticed 🤣), but we are going to try to fix that in Bug 1851244.
I appreciate the help getting to the bottom of the whole "virtual digitizer" thing too, but I'm a bit concerned about your suggested fix. If we filter out the \\??\\VIRTUAL_DIGITIZER device, doesn't that mean that the CSS Media Queries would not be responsive anymore when doing RDM testing? Or is there something else I'm missing?
Could we fix the problem by making sure we delete our virtual digitizer when we're done with it?
|
Reporter | |
Comment 11•9 months ago
|
||
If we filter out the
\\??\\VIRTUAL_DIGITIZERdevice, doesn't that mean that the CSS Media Queries would not be responsive anymore when doing RDM testing? Or is there something else I'm missing?
My recollection is that GetPointerCapabilities in layout/style/nsMediaFeatures.cpp (currently located at line 359) is responsible for checking the pointer mode. I'd linked this above, but the link broke as code changes were made (I probably shoulda seen that coming). The function in question always returns a coarse pointer if it's able to get the current browsing context and if that context's TouchEventsOverride() is enabled. Per comments in GetPointerCapabilities, the override is only enabled during RDM. So RDM should already be forcing a coarse pointer for media queries whether or not the digitizer is active and whether or not Firefox cares about it.
That said: I can't speak definitively. I don't actually work on Firefox, don't know how to download and build it from source, and don't have time to test. As far as I can remember, this bug report was the first and only time I've ever dug into Firefox's code. Anyone in a position to make these changes should be in a position to test them too, I'd expect.
Could we fix the problem by making sure we delete our virtual digitizer when we're done with it?
I'd expect that to improve things as well.
There's the caveat that I think Firefox not in RDM might still react to a virtual digitizer spawned by another process (the clearest example, if an uncommon one in practice, would be running two Firefoxes alongside each other with only one in RDM). However, deleting the digitizer properly would fix the "turning RDM on affects media queries until Firefox is shut down entirely" issue, which seems to me like the worst aspect of this whole bug.
|
||
Comment 12•9 months ago
|
||
(In reply to cob.bzmoz from comment #11)
My recollection is that
GetPointerCapabilitiesinlayout/style/nsMediaFeatures.cpp(currently located at line 359)
(Archaeological note: permalink to current revision.)
I'd linked this above, but the link broke as code changes were made (I probably shoulda seen that coming).
For future reference, just in case: if you press Y, or click 'Permalink' under the Navigation menu to the right, you can get a link like the above which includes the relevant commit hash, and therefore won't go stale. (Not as quickly, at least.)
Description
•