1563640 - SwiftKey keyboard doesn't work correctly on textboxes

Status: VERIFIED FIXED

(GeckoView :: IME, defect, P3)

Description

[Karl Dubost💡 :karlcow]

This was initially reported on https://webcompat.com/issues/33499

Requirements: Install swiftkey Keyboard on Firefox Android.

1. Go to https://mobile.twitter.com/home
2. Type foo
3. then type enter to add lines.

Actual:
lines are added but after the cursor.

Expected:
lines are added before the cursor insertion point. So when someone type again text, lines have been inserted between the two texts.

Comment 1

[Makoto Kato [:m_kato]]

SwiftKey seems to send key event (onKey* method) when tapping enter key. Most IME doesn't send this event. When this situation, 2nd onSelectionChange calls with old position. I guess that Gecko side's selection is updated yet, so it still uses old position.

07-05 11:22:45.551 17158 17176 I GeckoEditableSupport: IME: NotifyIMEOfTextChange: s=5, oe=5, ne=6
07-05 11:22:45.561 17158 17176 I GeckoEditableSupport: IME: NOTIFY_IME_OF_SELECTION_CHANGE
07-05 11:22:45.561 17158 17176 D GeckoEditable: onTextChange("↲", 5, 5)
07-05 11:22:45.561 17158 17176 D GeckoEditable: onSelectionChange(6, 6)
07-05 11:22:45.561 17158 17176 D GeckoEditable: reply: Action(TYPE_EVENT)
07-05 11:22:45.562 17158 17176 D GeckoEditable: onDefaultKeyEvent(action=1, keyCode=66, metaState=0, time=10218254, repeatCount=0)
07-05 11:22:45.562 17158 17176 D GeckoEditable: reply: Action(TYPE_EVENT)
07-05 11:22:45.562 17158 17176 D GeckoEditable: reply: Action(TYPE_REMOVE_SPAN)
07-05 11:22:45.562 17158 17176 I GeckoEditableSupport: IME: NOTIFY_IME_OF_SELECTION_CHANGE
07-05 11:22:45.562 17158 17176 D GeckoEditable: onSelectionChange(5, 5)

Comment 2

[Chris Peterson [:cpeterson]]

Adding [geckoview:fenix:m8] whiteboard tag because Makoto says he will work on this bug in Q3.

Comment 4

[Makoto Kato [:m_kato]]

icMaybeSendComposition will set selection during key event handling. Then Gecko is also updating text and selection. so this is race of both setting selection. So we should ignore to call onImeUpdateComposition if no composition.

Comment 5

[Makoto Kato [:m_kato]]

Attachment: Bug 1563640 - Swiftkey keyboard doesn't work correctly on textboxes.

This issue is race condition of Gecko thread and InputConnection thread.

When inputting [ENTER] in VKB, Switftkey generates keyboard event (down and up), then set empty span to current position.

It means that the following occurs.

1. Inserting CR by [ENTER] key event of Swiftkey.
2. Swiftkey sets empty span with current selection position. Then GV sets new selection with it due to adding span.
3. By 1., text and selection are updated, then GV receives new selection position by 1.
4. Selection notification by 2. is received, then selection is back to previous position unfortunately.

Although we should use 1 and 3's selection, GV uses 4's selection since this is last notification. But 2's selection is current selection until we don't update text. So it is unnecessary to set same selection again by 2.

Also, most IMEs don't send key event by 1, and they replace with new text without 1 and 2 So this issue occurs on Switftkey only.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1e44ecace5e5
Swiftkey keyboard doesn't work correctly on textboxes. r=geckoview-reviewers,snorp

Comment 7

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/1e44ecace5e5

Comment 8

[Chris Peterson [:cpeterson]]

Makoto, does this SwiftKey bug affect both GeckoView and Fennec? Should we uplift this fix to Fennec ESR 68?

Comment 9

[Makoto Kato [:m_kato]]

(In reply to Chris Peterson [:cpeterson] from comment #8)

Makoto, does this SwiftKey bug affect both GeckoView and Fennec? Should we uplift this fix to Fennec ESR 68?

Before uplifting this, I need fix bug 1572728 that is robocop test bug.

Comment 10

[Makoto Kato [:m_kato]]

Comment on attachment 9083541 [details]
Bug 1563640 - Swiftkey keyboard doesn't work correctly on textboxes.

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: When using SwiftKey on Firefox for Android, user cannot input text in <textarea> at current caret position. Firefox for Android doesn't release 69+ until Fenix is released, no one gets the fix on release channel.
• User impact if declined: When using SwiftKey on Firefox for Android, user cannot input text in <textarea> at current caret position.
• Fix Landed on Version: 70
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): We set invalid character offset that is meanless. This change ignores it. Also, this fix requires bug 1572728 to fix test failure.
• String or UUID changes made by this patch: No

Comment 11

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9083541 [details]
Bug 1563640 - Swiftkey keyboard doesn't work correctly on textboxes.

Fixes text input issues with Swiftkey. Approved for Fennec 68.2b2.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr68/rev/55ce33264266

Comment 13

[Diana Rus]

Hi, verified as fixed with Google Pixel 3a XL(Android 9), Samsung Galaxy S10+(Android 9) and Sony Xperia Z5 (Android 7) on Firefox Release 68.2 and Firefox Beta 68.2b7.
Verified the comments/text area element for:

• https://mobile.twitter.com/home
• https://news.ycombinator.com/
  I will mark firefox-esr68 as verified.

Comment 14

[Sorina Florean [:sflorean]]

Verified on Firefox Preview GV 70.0 - 2.1.0 with Nokia 6 (Android 7.1.1) and I can confirm that the issue is not reproducible.

Comment 15

[Chris Peterson [:cpeterson]]

Moving some keyboard bugs to the new GeckoView::IME component.