1890522 - Include possibble CSS scaling when computing coordinates in nsContentUtils::SendMouseEvent

Status: NEW

(Core :: Panning and Zooming, defect, P3)

Description

[Alex Jakobi [:ajakobi]]

Found during bug 1733509 when using EventUtils.js::synthesizeMouse in mochitest for test_event_target_radius.html.

We have an element with a CSS scale of 0.5 that has a child in-process iframe. When performing event re-targeting tests for that iframe, it became apparent that the coordinate translation of nsContentUtils::SendMouseEvent (most likely in nsContentUtils::ToWidgetPoint) is not taking possible CSS scaling of parent elements into account and therefore performing clicks at wrong positions.

Comment 1

[Hiroyuki Ikezoe (:hiro)]

Looks like the presShell used in ToWidgetPoint is the in-process iframe's.

I suppose SendMouseEvent in in-process iframes doesn't work with visualViewport.zoom != 1.0 cases either.

Comment 2

[Botond Ballo [:botond]]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #1)

I suppose SendMouseEvent in in-process iframes doesn't work with visualViewport.zoom != 1.0 cases either.

I believe that should be handled as the pinch-zoom scale is included in the layout-to-visual transform applied here.

Comment 3

[Hiroyuki Ikezoe (:hiro)]

Yeah, the place what I meant. The presContext is an in-process one. And IIUC the root scroll frame for the in-process presShell is not the top level one.

Comment 4

[Botond Ballo [:botond]]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #3)

Yeah, the place what I meant. The presContext is an in-process one. And IIUC the root scroll frame for the in-process presShell is not the top level one.

LayoutToVisual() will apply the pinch-zoom scale even if the presContext passed in is a subdocument pres context (the implementation queries the RCD to get the resolution here).

Comment 5

[Hiroyuki Ikezoe (:hiro)]

Huh, that's confusing.

Comment 6

[Botond Ballo [:botond]]

Capturing some relevant Matrix discussion with Timothy:

[botond] Is it a known issue that nsView::GetNearestWidget() does not handle CSS scale transforms in its computation of the aOffset out-parameter? [...]
[tnikkel] yeah thats known. apis like that and nsIFrame::GetOffsetTo don't take into account transforms. if transforms are important to take into account another api should be used
[...]
[botond] Do you have a suggestion for a better API to use in this case?
[tnikkel] get the point relative to the root frame of the subdoc, convert it from that frame to be relative to the root frame of the root doc, then convert to be widget relative
[tnikkel] this is basically doing the inverse of GetEventCoordinatesRelativeTo https://searchfox.org/mozilla-central/rev/1f27a4022f9f1269d897526c1c892a57743e650c/layout/base/nsLayoutUtils.cpp#1545
[tnikkel] this code is sort of doing that i think https://searchfox.org/mozilla-central/rev/1f27a4022f9f1269d897526c1c892a57743e650c/layout/base/PositionedEventTargeting.cpp#571

Comment 7

[Timothy Nikkel (:tnikkel)]

The short answer is use nsLayoutUtils::GetTransformToAncestor or one of the functions that uses it to transform between frames to do the bulk of the work, and then on either end you do a little conversions to/from view, widget, or presshell relative coords.

Comment 9

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

This also affects to nsIDOMWindowUtils.sendTouchEvent because it uses nsContentUtils::ToWidgetPoint too. The API is used by the touch emulation of the responsive design mode of DevTools. So, this is not an API only bug, users may see this bug.