1691622 - Cannot synthesize native middle/right mouse button event via `nsIDOMWindowUtils::SendNativeMouseEvent()` on Linux

Status: RESOLVED FIXED

(Core :: DOM: UI Events & Focus Handling, defect, P3)

Description

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

The API was defined with assuming that a native event is mapped to a mouse button down and up. However, on Linux and Android (and 4th or later buttons on Windows) do not so. They just have "mousedown" or "mouseup" message and specifies additional information of mouse button ID.

Comment 1

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

I'll fix this only for Linux for now.

Comment 2

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 1: Make `synthesizeNativeMouseClick` take an `Object` instead of multiple arguments for synthesizing events r=smaug

In these days, API should take an Object instead of multiple arguments
since the callers look like using "named" arguments and this allows to
add new optional arguments with changing not all callers.

This patch also changes similar API for APZ aware tests for keeping
consistent style for their users.

Comment 3

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 2: Merge `synthesizeNativeMouseClickAtCenter` with `synthesizeNativeMouseClick` r=smaug

With adding new feild to the aParam, synthesizeNativeMouseClick can
work as synthesizenativeMouseClickAtCenter too. Therefore, we can get
rid of the redundant API.

Additionally, this patch makes synthesizeNativeMouseClickWithAPZ can
take the new field for consistency between them.

Depends on D105755

Comment 4

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 3: Make `synthesizeNativeMouseClick` take screenX/Y r=smaug

Some tests want to specify screen position directly. Therefore, they should
have additional fields in the Object param to take offset in screen.

Depends on D105756

Comment 5

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 4: Make `synthesizeNativeMouseClick*()` aware of modifiers r=smaug

Surprisingly, they don't take modifiers, and
nsIWidget::SynthesizeNativeMouseEvent() which are implementations of
nsIDOMWindowUtils::SendNativeMouseEvent() treat given modifier flags
are native's ones, and handle modifiers only on macOS. Therefore, this
patch makes them handle native modifiers of Gecko.

Unfortunately, I'm not so familiar with Android API, and in the short
term, I don't need the support on Android. Therefore, this patch just
adds a TODO comment on Android widget.

Additionally, we don't have a simple way to set modifier only while
posting a mouse input on Windows too. It requires complicated code.
Therefore, I don't add the support for it on Windows too.

Depends on D105757

Comment 6

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 5: Make `synthesizeNativeMouseClick*` take an event type r=smaug

Let's make synthesizeNativeMouseClick* take an event type, and only when
it's click, it should send native mouse event twice.

Then, we can all them synthesizeNativeMouseEvent*.

Depends on D105758

Comment 7

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 6: Merge `synthesizeNativeMouseClick*` with `synthesizeNativeMouseEvent*` r=smaug

Depends on D105759

Comment 8

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 7: Merge `synthesizeNativeMouseMove*` with `synthesizeNativeMouseEvent*` r=smaug

Depends on D105760

Comment 9

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 8: Get rid of `nsIDOMWindowUtils::SendNativeMouseMove()` r=smaug

Now, there are no users of this API. However,
nsIWidget::SynthesizeNativeMouseMove() is still used by EventStateManager.
Even though it's just redirected to nsIWidget::SynthesizeNativeMouseEvent(),
but it hides the native event message from EventStateManager. Therefore,
this patch keeps the widget API for now.

Depends on D105761

Comment 10

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 9: Make `nsIWidget::SynthesizeNativeMouseEvent` take an XP button ID and abstract message value r=smaug

Currently, it takes a raw native message value, but it makes JS content too
complicated. And on Linux, it cannot synthesize non-primary button events
because GDK has only button press and release messages which dont' include
mouse button information.

For solving these problems, this patch creates a new abstract native message
as nsIWidget::NativeMouseMessage and makes each widget converts it to
a platform native message.

Additionally, this patch adds an argument to make it possible its callers
to specify pressing or releasing mouse button with a DOM mouse button value.

Note that the following patch adds new argument to
synthesizeNativeEventMouse* for mochitests and which will be tested by
new tests.

Depends on D105762

Comment 11

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 10: Make `synthesizeNativeMouseEvent` take a button argument r=smaug

Now, it should take button argument whose default value is 0 (primary
button).

Depends on D105763

Comment 12

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 11: Make mochitests stop using `nsIDOMWindowUtils.sendNativeMouseEvent` directly as far as possible r=smaug

For making the test framework/API change easier, such raw API shouldn't be
used directly. Therefore, this patch makes tests using it directly stop
using it and use synthesizeNativeMouseEvent instead.

However, this patch does not fix browser_touch_event_iframes.js because
it accesses the API from ContentTask. So, EventUtils.js isn't available
without larger change.

Depends on D105764

Comment 13

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 12: Change the default unit of `screenX/Y` of `synthesizeNativeMouseEvent` from device pixels to CSS pixels r=smaug

For minimizing the previous patch changes, scale's default value for
screenX/Y is treated as 1.0. It means that screenX/Y are device
pixels by default, but offsetX/Y are in CSS pixels by default. This
difference may make developers confused. Therefore, we should align the
default unit of them to screenPixelsPerCSSPixel. I.e., their default
unit becomes CSS pixels.

Depends on D105765

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e649d8ff7787
part 1: Make `synthesizeNativeMouseClick` take an `Object` instead of multiple arguments for synthesizing events r=smaug
https://hg.mozilla.org/integration/autoland/rev/301ce838d678
part 2: Merge `synthesizeNativeMouseClickAtCenter` with `synthesizeNativeMouseClick` r=smaug
https://hg.mozilla.org/integration/autoland/rev/54b426637c32
part 3: Make `synthesizeNativeMouseClick` take screenX/Y r=smaug
https://hg.mozilla.org/integration/autoland/rev/727097623436
part 4: Make `synthesizeNativeMouseClick*()` aware of modifiers r=smaug,geckoview-reviewers,m_kato
https://hg.mozilla.org/integration/autoland/rev/d21beb190305
part 5: Make `synthesizeNativeMouseClick*` take an event type r=smaug
https://hg.mozilla.org/integration/autoland/rev/dc5e230ba557
part 6: Merge `synthesizeNativeMouseClick*` with `synthesizeNativeMouseEvent*` r=smaug
https://hg.mozilla.org/integration/autoland/rev/6af85d93a7f2
part 7: Merge `synthesizeNativeMouseMove*` with `synthesizeNativeMouseEvent*` r=smaug
https://hg.mozilla.org/integration/autoland/rev/b3e0851f5cec
part 8: Get rid of `nsIDOMWindowUtils::SendNativeMouseMove()` r=smaug
https://hg.mozilla.org/integration/autoland/rev/1008909e3dde
part 9: Make `nsIWidget::SynthesizeNativeMouseEvent` take an XP button ID and abstract message value r=smaug,geckoview-reviewers,agi,m_kato
https://hg.mozilla.org/integration/autoland/rev/5fc38c58de82
part 10: Make `synthesizeNativeMouseEvent` take a button argument r=smaug
https://hg.mozilla.org/integration/autoland/rev/da33e4b8fa18
part 11: Make mochitests stop using `nsIDOMWindowUtils.sendNativeMouseEvent` directly as far as possible r=smaug
https://hg.mozilla.org/integration/autoland/rev/ed9c8bd8966c
part 12: Change the default unit of `screenX/Y` of `synthesizeNativeMouseEvent` from device pixels to CSS pixels r=smaug

Comment 15

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1691622 - part 13: Make `helper_scrollbarbutton_repeat.html` use new API to synthesize native mouse click

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f3050f40efe5
part 13: Make `helper_scrollbarbutton_repeat.html` use new API to synthesize native mouse click. r=fix CLOSED TREE

Comment 17

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/e649d8ff7787
https://hg.mozilla.org/mozilla-central/rev/301ce838d678
https://hg.mozilla.org/mozilla-central/rev/54b426637c32
https://hg.mozilla.org/mozilla-central/rev/727097623436
https://hg.mozilla.org/mozilla-central/rev/d21beb190305
https://hg.mozilla.org/mozilla-central/rev/dc5e230ba557
https://hg.mozilla.org/mozilla-central/rev/6af85d93a7f2
https://hg.mozilla.org/mozilla-central/rev/b3e0851f5cec
https://hg.mozilla.org/mozilla-central/rev/1008909e3dde
https://hg.mozilla.org/mozilla-central/rev/5fc38c58de82
https://hg.mozilla.org/mozilla-central/rev/da33e4b8fa18
https://hg.mozilla.org/mozilla-central/rev/ed9c8bd8966c
https://hg.mozilla.org/mozilla-central/rev/f3050f40efe5