1648491 - [touchscreen] Pinch zoom on map is not working on flightradar24.com,while using Direct3D compositing

Status: RESOLVED FIXED

(Core :: Panning and Zooming, defect, P3)

Description

[Gyula Palko[:gpalko]]

Affected versions
Nightly 79.0a1

Affected platforms
Devices with touchscreen (tested on win10- Lenovo yoga)

Preconditions
apz.allow_zooming = true
apz.windows.use_direct_manipulation = true
gfx.webrender.force-disabled = true

Steps to reproduce

1. Open https://flightradar24.com
2. Initiate pinch zooming on the map using the touchscreen

Expected result
APZ zooming is performed

Actual Result
The map view doesn't react to pinch gesture

Note
Pinch zoom on the other elements from the page is working as expected(title bar, side panel. buttons)
Pinch zoom initiated from the touchpad is working as expected

Comment 1

[Timothy Nikkel (:tnikkel)]

I tested this on Win 8.1 with Surface Pro 2 and the map zoomed as I expected.

Comment 2

[Gyula Palko[:gpalko]]

I can't reproduce it on Surface Pro with win10, but is still reproducible on Lenovo Yoga(win10, ARM based)

Comment 3

[Timothy Nikkel (:tnikkel)]

I wonder if the site is sending different content based on the UA string? Could you spoof the Surface Pro UA string on the Lenova Yoga and see if it works?

Comment 4

[Gyula Palko[:gpalko]]

I've used the UA string from Surface on Yoga, the issue is still reproducing.

Comment 5

[Kris Taeleman (:ktaeleman)]

Tested on Win10 Dell XPS 9570, works as expected.

Comment 6

[Timothy Nikkel (:tnikkel)]

I think kats has a touchscreen arm laptop, maybe he can reproduce. That would tell us if this is ARM related or something else.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

I do have one, I can try on Tuesday. (On PTO Monday)

Comment 8

[Kartikaya Gupta (email:[e-mail])]

I can reproduce on the Lenovo. Does seem like a bug, since touch-pinching works in Chrome. Also doing a one-touch-pinch gesture in Firefox seems to have some effect although it's unclear to me at first glance if that's implemented in-content or if it's APZ sending ctrl+wheel events to the page and triggering the effect. The page is pretty laggy so it's hard to tell just by observing the behaviour, but at least I can repro and eventually debug it.

Comment 9

[Gyula Palko[:gpalko]]

I've just noticed, that this isn't reproducing with WebRender ON on Lenovo. On Surface, WebRender is ON by default, that's why you weren't able to reproduce it. Using Direct3D composition, can be reproduced on both devices.

Comment 10

[Kartikaya Gupta (email:[e-mail])]

I started looking into this. It looks like the content has a touch-action: pan-x pan-y on the element and then uses pointer events do implement zooming. However in FF with allow_zooming=true we don't seem to send any pointermove events for a two-finger gesture. Not sure why yet but it's definitely a bug we need to fix.

Comment 11

[Kartikaya Gupta (email:[e-mail])]

The problem is that the area with the touch-action ends up in a dispatch-to-content region, so APZ errs on the side of marking the input events as consumable here which then causes the pointer events being sent to content to be cancelled. Exactly as described in the comment here.

I have a simple page where I can reproduce the problem so I'll try to dig into why the layer tree ends up with a d-t-c region when I think it should be able to have a more accurate representation.

Comment 12

[Kartikaya Gupta (email:[e-mail])]

The code here is intended to collapse touch-action regions into a d-t-c region when we have multiple elements that are going onto the same layer. But it ends up collapsing a single element with pan-x pan-y because the block above, here adds to both the vertical and horizontal rects, which then gets detected as "multiple" elements. The fix is pretty simple, but I need to verify it fixes the original problem too.

Comment 13

[Kartikaya Gupta (email:[e-mail])]

Unfortunately the simple fix didn't work. The problem seems to be that if you have pan-x pan-y set on a div, all the descendants also inherit that behaviour, and when the display items for those descendant elements get squashed into a layer, this gets detected as conflicting information and everything gets collapsed into a d-t-c region. I think we need to propagate a bit more information through the display item to indicate that these are all "compatible" touch action regions in that they came from a single frame's touch-action property, and it should be safe to squash them together.

Comment 14

[Kartikaya Gupta (email:[e-mail])]

Sigh. I wrote a fix for the problem in the previous comment, but it turns out there's also a wheel listener on the element. As that's an APZ-aware listener, it forces the d-t-c region anyway. I don't know if this is easily fixable without adding a new region type to the layer event regions.

I'll spin out a dependency for the fix that I wrote since that might be worth landing anyway.

Comment 15

[Kartikaya Gupta (email:[e-mail])]

Botond suggested doing a more thorough check on the main thread here to maybe avoid sending that pointercancel in this scenario. That seems like a better approach so I'll look into that.

Comment 16

[Kartikaya Gupta (email:[e-mail])]

That approach works. Try push including a test https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=fcb7d231b136c08c80c0f32fda14e5bcba426472

There's an intermittent failure on Android which is surprising because the test is an almost-exact copy of a pre-existing test. But I'll try reproducing that locally.

Comment 17

[Kartikaya Gupta (email:[e-mail])]

I did another try push with some more logging and it's looking like it might be a bug in the pointer events implementation. As far as I can tell the event is being synthesized and passed through to gecko fine, but gecko isn't notifying the content listener for the last pointerup event.

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=314812905&repo=try&lineNumber=1729

Comment 18

[Kartikaya Gupta (email:[e-mail])]

Ah, turns out with pan-x pan-y two finger "zoom" attempts do still end up panning, so the last pointerup ends up on the body rather than the target element. I think this behaviour is reasonable, just need to update the test accordingly.

Comment 19

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1648491 - Have the main thread double-check APZ's consumable state. r?botond

APZ can sometimes indicate that it will be consuming touch events, even though
the touch-action properties prohibit it. This can happen if, for example, APZ
is waiting on the main-thread for accurate touch-action information. In such
cases, the main thread has enough information to filter out these false positives.
This patch makes it do that, by plumbing the allowed touch behaviors into
the APZEventState code that triggers the pointercancel event.

Comment 20

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1648491 - Add a test. r?botond

Depends on D89303

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ca7ab0e42f54
Have the main thread double-check APZ's consumable state. r=botond
https://hg.mozilla.org/integration/autoland/rev/27f5e77cd9bb
Add a test. r=botond

Comment 22

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/ca7ab0e42f54
https://hg.mozilla.org/mozilla-central/rev/27f5e77cd9bb

Comment 23

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

Comment 24

[Kartikaya Gupta (email:[e-mail])]

This only affects configurations with apz.allow_zooming=true, which is not the default on versions prior to 82.