1776843 - Investigate behavior on touch devices with Gecko-based non-Android OSs

Status: RESOLVED FIXED

(Core :: DOM: Copy & Paste and Drag & Drop, task)

Description

[Mirko Brodesser (:mbrodesser-Igalia)]

OSs include: Linux, Windows, macOS. Excluded: iOS (since Firefox there isn't Gecko-based).

With current Nightly.

First, manually test it by:
a) Flipping the pref "dom.events.asyncClipboard.readText" to true.
b) Open https://jsfiddle.net/68hng59s/.
c) Touch the "X" button and check whether a "Paste" button occurs. Be aware of bug 1766803.
d) Touch the "Paste" button. An alert dialog with the clipboard's text content should appear.

Secondly, potentially write automated tests for it.

Comment 1

[Cristian Baica [:cbaica], Release Desktop QA]

Manually tested using the latest nightly (Fx104.0a1) on the following machines:

• Intel Iris Xe: Windows 11 and Ubuntu 22.04 - Touching the 'Paste' button correctly prompts the clipboard text inside the alert box.
• Yoga: Windows 10 ARM - Touching the 'Paste' button correctly prompts the clipboard text inside the alert box.

As far as I can tell, there are no touchscreen mac devices, except iOS devices.

Comment 2

[Mirko Brodesser (:mbrodesser-Igalia)]

Cristian: could you please repeat that, record the screen (the browser window suffices) and attach the recording here? Sorry that I didn't ask about that before. Either of the Windows systems plus Ubuntu suffices.

As far as I can tell, there are no touchscreen mac devices, except iOS devices.

Okay, thanks.

Comment 3

[Cristian Baica [:cbaica], Release Desktop QA]

Attachment: windows 11

Comment 4

[Cristian Baica [:cbaica], Release Desktop QA]

Attachment: ubuntu 22

Here are the recordings for Ubuntu 22.04 and Windows 11. I couldn't get the recording in one file, so I hope this is ok.

Comment 5

[Mirko Brodesser (:mbrodesser-Igalia)]

(In reply to Cristian Baica [:cbaica], Release Desktop QA from comment #4)

Created attachment 9284093 [details]
ubuntu 22

Here are the recordings for Ubuntu 22.04 and Windows 11. I couldn't get the recording in one file, so I hope this is ok.

Thanks, that's OK.

Comment 6

[Mirko Brodesser (:mbrodesser-Igalia)]

Interestingly, the "Paste" button's position is wrong in the Windows-recording (https://bugzilla.mozilla.org/attachment.cgi?id=9284092). @Cristian: in that recording, how many physical screens did you use?

CCing Neil, because that issue might be related to bug 1766803.

Comment 7

[Makoto Kato [:m_kato]]

nsIDOMWindowUtils.getLastOverWindowMouseLocationInCSSPixels records real mouse event only, so we cannot use it on parent process even if Android. Child process can receive converted mouse event from touch event, it works well on child process only.

Comment 8

[Mirko Brodesser (:mbrodesser-Igalia)]

(In reply to Makoto Kato [:m_kato] from comment #7)

nsIDOMWindowUtils.getLastOverWindowMouseLocationInCSSPixels records real mouse event only, so we cannot use it on parent process even if Android. Child process can receive converted mouse event from touch event, it works well on child process only.

Makoto: do you mean nsIDOMWindowUtils.getLastOverWindowMouseLocationInCSSPixels provides no touch position if called from the parent process, but it does provide a touch position if called from the child process? If so, I don't understand why the manual test on Ubuntu 22 worked.

Comment 9

[Makoto Kato [:m_kato]]

Makoto: do you mean nsIDOMWindowUtils.getLastOverWindowMouseLocationInCSSPixels provides no touch position if called from the parent process, but it does provide a touch position if called from the child process? If so, I don't understand why the manual test on Ubuntu 22 worked.

Yes on Android. Although I would like to use it on parent process, this value on parent process isn't updated as long as I test. I guess that this is as same as Windows. I guess that we dispatch converted event (original is touch event?) to child process, then it is recorded.

Although I have no test environment of GTK touch device, I guess that GTK dispatches anything mouse event? even if touch. But I am not sure.

Comment 10

[Mirko Brodesser (:mbrodesser-Igalia)]

(In reply to Makoto Kato [:m_kato] from comment #9)

Makoto: do you mean nsIDOMWindowUtils.getLastOverWindowMouseLocationInCSSPixels provides no touch position if called from the parent process, but it does provide a touch position if called from the child process? If so, I don't understand why the manual test on Ubuntu 22 worked.

Yes on Android. Although I would like to use it on parent process, this value on parent process isn't updated as long as I test.

Thanks, understood.

I guess that this is as same as Windows. I guess that we dispatch converted event (original is touch event?) to child process, then it is recorded.

I'm not familiar with the code responsible for that. Perhaps Edgar can provide some pointers and potentially context?

Although I have no test environment of GTK touch device, I guess that GTK dispatches anything mouse event? even if touch. But I am not sure.

Comment 11

[Edgar Chen [:edgar] (Parental leave until Oct. 09.)]

(In reply to Makoto Kato [:m_kato] from comment #9)

I guess that this is as same as Windows. I guess that we dispatch converted event (original is touch event?) to child process, then it is recorded.

For Windows the synthesized mouse event is fired from APZ, and yes, it seems in parent process there is no synthesized mouse event, I filed bug 1778486.

Comment 12

[Cristian Baica [:cbaica], Release Desktop QA]

(In reply to Mirko Brodesser (:mbrodesser) from comment #6)

@Cristian: in that recording, how many physical screens did you use?

Hey Mirko. Sorry for the late response. Only 1 physical screen was used in this test. I didn't mention anything about the position of the paste button, because, as you noted, it's related to bug 1766803.

Comment 13

[Edgar Chen [:edgar] (Parental leave until Oct. 09.)]

I am going to write a test for touch input.

Comment 14

[Edgar Chen [:edgar] (Parental leave until Oct. 09.)]

Attachment: Bug 1776843 - Part 1: Move clipboard tests to dom/events/test/clipboard/;

Comment 15

[Edgar Chen [:edgar] (Parental leave until Oct. 09.)]

Attachment: Bug 1776843 - Part 2: Add clipboard contextmenu test for touch input;

Depends on D152653

Comment 16

[Edgar Chen [:edgar] (Parental leave until Oct. 09.)]

https://treeherder.mozilla.org/jobs?repo=try&revision=eeb0ca7354bb3a05350c4c4a7efa5eedcbf7d02f

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/11158f1eb5a0
Part 1: Move clipboard tests to dom/events/test/clipboard/; r=smaug
https://hg.mozilla.org/integration/autoland/rev/8fb31906e84c
Part 2: Add clipboard contextmenu test for touch input; r=smaug

Comment 18

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/11158f1eb5a0
https://hg.mozilla.org/mozilla-central/rev/8fb31906e84c