1866431 - "WebDriver:ElementSendKeys" fails to send text that contains surrogate pairs

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Originally filed as: https://github.com/mozilla/geckodriver/issues/2139

Note that this problem only occurs when running Nightly and early beta builds of Firefox. Using mozregression bug 1840519 has been revealed.

Note that this happens only in these builds of Firefox because there the preference dom.event.keypress.dispatch_once_per_surrogate_pair is set to false. If it's true the following Marionette testcase will pass:

    def test(self):
        self.marionette.set_pref(
            "dom.event.keypress.dispatch_once_per_surrogate_pair", False)

        self.marionette.navigate(
            "https://www.selenium.dev/selenium/web/cn-test.html")
        elem = self.marionette.find_element(By.NAME, "i18n")
      
        text = "𠀀𠜎𠀋𪆐𪚲"
        elem.send_keys(text)
        self.assertEqual(elem.get_property("value"), text)

Masayuki, can you please take a look? For now I filed it as a Marionette bug but I assume it's somewhere in DOMWindowUtils or platform code?

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Marionette testcase (patch)

This testcase can easily be run after applying with the following command:

./mach marionette-test --headless -vv --gecko-log - --setpref="remote.log.truncate=true" testing/marionette/harness/marionette_harness/tests/unit/test_typing.py

Comment 2

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

I guess that the Marionette synthesizes different style of keypress events.

It seems that here splits surrogate pairs to surrogate:
https://searchfox.org/mozilla-central/rev/edb2612db13e89f1c44ab95b1e4d4366c16eb9fb/testing/marionette/client/marionette_driver/marionette.py#188-190

However, I'm not familiar with Python and its string so that I'm not 100% sure whether here is buggy, and I'm not sure whether Python can check whether a surrogate or not for each "character" smoothly.

Another possible fix point (if here splits surrogate pairs) is, at the receiver side. If it gets a high-surrogate keydown and/or keyup event, make it wait for the following keydown and/or keyup for a low-surrogate. Then, they can synthesize a set of keys with checking the pref. I guess that this is better solution, but I've not found the receiver side. Could you let me know where is it?

(I.e., a surrogate pair will synthesize a set of keydown and keyup with one or two keypress events once the new behavior will be shipped and that's what same behavior as user input at least on Windows.)

And what do you think for this comment from your point of view?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Discussed in todays triage meeting and James will have a look. If it's a problem with Marionette we might turn off this new feature for now until we got it correctly implemented in Marionette.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Just two additional maybe related issues that I noticed:

• Broken WPT tests for multiple Unicode scalar values: https://github.com/web-platform-tests/wpt/pull/40785
• IME support in actions: https://github.com/w3c/webdriver/issues/1683

Comment 5

[James Graham [:jgraham]]

Attachment: Bug 1866431 - Fixture for setting prefs in mozilla-only wdspec tests,

This is mostly reusing the implementation from the marionette client.

Comment 6

[James Graham [:jgraham]]

Attachment: Bug 1866431 - Send full codepoints when creating key events,

Previously, given a string in element send keys, we iterated over each
UCS2 "character" in the string, so that non-BMP characters ended up as
separate key events, one for the high surrogate and one for the low
surrogate.

With this change we instead use the built in string iterator which
provides the surrogate pair in the case of non-BMP characters, so we
emit a single event for each unicode codepoint.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b2886b896d0d
Fixture for setting prefs in mozilla-only wdspec tests, r=webdriver-reviewers,whimboo

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The second patch also needs to be pushed. As such lets mark as leave-open for now.

Comment 9

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/b2886b896d0d

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f19c3570bc57
Send full codepoints when creating key events, r=webdriver-reviewers,masayuki,whimboo

Comment 11

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/43722 for changes under testing/web-platform/tests

Comment 12

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/f19c3570bc57

Comment 13

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:jgraham, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox122 to wontfix.

For more information, please visit BugBot documentation.

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The fix here on this bug is actually for a feature that is only enabled in Firefox Nightly. So beta is not affected.