1508200 - Google Translate keyboard duplicates accented characters

Status: RESOLVED WORKSFORME

(Web Compatibility :: Site Reports, defect, P1)

Description

[Markus Popp]

Attachment: Screenshot showing duplicate entry

I found this in Firefox 63 on Linux64 (fresh profile, no addons), it may apply to other platforms as well.

To reproduce:

* go to https://translate.google.com/#cs/en/

* activate the Czech keyboard (1st item in dropdown)

* enter á (accent key + a, I did this with a German keyboard)

* text area displays aá (normal a followed by a with accent)

The same happens with other accented characters. I also tried this in Google Chrome 70, there I only get à as expected.

Comment 1

[:Gijs (he/him)]

(In reply to Markus Popp from comment #0)
> * enter á (accent key + a, I did this with a German keyboard)

Reporter: Is this with dead keys turned on, or something? Otherwise, what do you mean by "accent key" ? Are you able to check if the same thing happens on nightly?

Masayuki-san, any chance this is related to our key handling changes?

Comment 2

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

If this is a new regression starting from 63, I have no idea. However, I'm working on fixing odd keydown event handling on Linux in bug 1505147. Here will be a patched build (I mean, I kicked to build opt now): https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=213076438&revision=cab0b2a047d8d3e319607ed85362fd4c0aec4d5c This *might* fix this bug too.

(Today, I'm in PTO, and tomorrow is Japanese national holiday. So, I'll be back next Monday.)

Comment 3

[Markus Popp]

(In reply to :Gijs (mostly not available until Monday; he/him) from comment #1)
> (In reply to Markus Popp from comment #0)
> > * enter á (accent key + a, I did this with a German keyboard)
> 
> Reporter: Is this with dead keys turned on, or something? Otherwise, what do
> you mean by "accent key" ? Are you able to check if the same thing happens
> on nightly?

Yes, this is with dead keys on. On a German keyboard like:

https://en.wikipedia.org/wiki/German_keyboard_layout#/media/File:KB_Germany.svg

to type 'á', I first press the 2nd key from right in the top row followed by 'a'. Which produces 'aá' in Google Translate.

The same happens in Firefox Nightly. However some time ago this didn't happen in the then-current Firefox. I can't remember exactly how long ago that was, maybe a few weeks or a few (not many) months. So the bug must be relatively new.

Comment 4

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

I cannot reproduce this bug with Ubuntu 18.04 (iBus), Ubuntu 18.10 (iBus) nor Debian 9 (Fcitx).

Do you still reproduce this bug? It may be a bug of Google Translate and that may have been fixed.

If you still reproduce this bug, could you let me know about your environment more? Distribution and active IME information are important.

Comment 5

[Markus Popp]

Attachment: keyboard1.png

Comment 6

[Markus Popp]

Attachment: keyboard2.png

Comment 7

[Markus Popp]

Attachment: keyboard3.png

Comment 8

[Markus Popp]

Attachment: ibus1.png

Comment 9

[Markus Popp]

Attachment: ibus2.png

Comment 10

[Markus Popp]

Attachment: ibus3.png

Comment 11

[Markus Popp]

Attachment: ibus4.png

Comment 12

[Markus Popp]

(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #4)
> I cannot reproduce this bug with Ubuntu 18.04 (iBus), Ubuntu 18.10 (iBus)
> nor Debian 9 (Fcitx).
> 
> Do you still reproduce this bug? It may be a bug of Google Translate and
> that may have been fixed.
> 
> If you still reproduce this bug, could you let me know about your
> environment more? Distribution and active IME information are important.

The issue is still there in the latest Firefox Nightly.

I attached screenshots of my keyboard and IBUS (1.5.17) settings. My distribution is Xubuntu 18.04 (64bit).

Comment 13

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Thanks! I'll try to create a Xubuntu 18.04 environment.

Comment 14

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Ah, I misunderstood. I understood as you are talking about direct inputting, but you reported software keyboard implemented by Google Translate. And I got it when I remove all keyboard layouts except German keyboard layout which has dead key, I reproduce it. When I try same key sequence in <https://w3c.github.io/uievents/tools/key-event-viewer.html>, I see redundant "keydown" event. This may causes the odd input.

Comment 15

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Hmm... Even if I make stop dispatching the odd keydown event (caused by odd behavior of the keyboard layout), I still see same symptom. Perhaps, Google Translate does something special when it runs on Gecko since we haven't dispatched "keydown" nor "keyup" event during composition.

Comment 16

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

This is caused by starting to dispatch "keydown" event and "keyup" event even during composition. So, this means that Google Translate has Gecko specific path for handling their virtual keyboard. So, we cannot do anything here.

I reproduce this bug with the following STR:

1. Create Ubuntu environment.
2. Make keyboard layout only "German (QWERTY)" keyboard layout in settings of Ubuntu (i.e., remove your daily-use keyboard layout).  This is really important step because Gecko refers primary keyboard layout at computing KeyboardEvent.keyCode value on Linux.
3. Open https://translate.google.com/#cs/en/
4. Click the keyboard icon and open Czech virtual keyboard.
5. Ensure focus in the textarea to input translated text.
6. Type "=" key of US keyboard layout.
7. Type "a".

Then, you'll see "aá" on Ubuntu 18.04, or "´a" on Ubuntu 18.10. Of course, it should be "á".

On Ubuntu 18.04, the following events are fired (bottom to top):
keyup: { keycode: 0x65, which: 0x65, key: 'a', code: 'KeyA' }
input
compositionend: { data: 'á' }
compositionupdate: { data: 'á' }
compositionstart
keydown: { keycode: 0x65, which: 0x65, key: 'a', code: 'KeyA' }
keyup: {keyCode: 0, which: 0, key: 'Dead', code: 'Euqual' }
keydown: {keyCode: 0, which: 0, key: 'Dead', code: 'Euqual' }

On Ubuntu 18.10, the following events are fired (bottom to top):
keyup: { keycode: 0x65, which: 0x65, key: 'a', code: 'KeyA' }
input
compositionend: { data: 'á' }
compositionupdate: { data: 'á' }
input
compositionupdate: { data: '´' }
compositionstart
keydown: {keyCode: 0, which: 0, key: 'Dead', code: 'Euqual' }

Only on Ubuntu 18.04, even if I disable the new behavior, I can reproduce same bug.

Mike Taylor:

Could you contact Google? The new behavior will be shipped with other keypress/window.event changes.

Comment 17

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(FYI: Of course, those events are completely valid.)

Comment 18

[Mike Taylor [:miketaylr]]

I sent an email to our Google partner list.

Comment 19

[Liz Henry (:lizzard) (relman/hg->git project)]

Tracking to make sure we check up on this during 65 beta.

Comment 20

[Panos Astithas (he/him) [:past] (please ni?)]

Assigning to Mike per comment 18.

Comment 21

[Ryan VanderMeulen [:RyanVM]]

Hey Mike, did we hear anything from Google on this yet?

Comment 22

[Mike Taylor [:miketaylr]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #21)

Hey Mike, did we hear anything from Google on this yet?

Nothing yet (from my message on Nov 30). I've just re-pinged the thread.

Comment 23

[Ryan VanderMeulen [:RyanVM]]

IIUC, the behavior changes were reverted for Fx65 via bug 1520756. We should definitely try to verify that b12 behaves better.

Comment 24

[Ryan VanderMeulen [:RyanVM]]

Mike tells me I was being overly-optimistic on that. That revert won't affect this behavior change.

Comment 25

[Mike Taylor [:miketaylr]]

Dennis, is this something we could fix via a webcompat system addon patch?

Comment 26

[Dennis Schubert [:denschub]]

FWIW, I can reproduce the STR from comment 16 even on macOS, when I set a German keyboard layout on my en-US MacBook, which then inserts a ;´ on pressing =. So this does not seem to be limited to Linux.

is this something we could fix via a webcompat system addon patch?

I poked at it for a bit, but I don't think we can. Although there seems to be a Gecko-specific path somewhere, it's not based on the User Agent which we could simply override. I'm having a hard time even locating the offending JS, but even if I discover the source, none of the input processing methods are exposed to a publicly writeable object, so modifying those sources is hard. The only way to theoretically change something here would be by replacing an entire JS file, which seems dangerous.

To resolve the ni? here, I guess the answer is no. I will dig a bit further, and although I don't expect to be able to find a way to build a WebCompat GoFaster patch here, I'll update this if I find something.

Comment 27

[Mike Taylor [:miketaylr]]

I've re-pinged our Google contact off-thread and asked for an update.

Comment 28

[Ryan VanderMeulen [:RyanVM]]

Hey Mike, any updates from Google on this?

Comment 29

[Mike Taylor [:miketaylr]]

I re-pinged jst, and nothing positive to report yet. :(

Comment 30

[Ryan VanderMeulen [:RyanVM]]

Pretty much a certainty that nothing's going to happen for 65 here unless Google changes something on their end.

Comment 31

[Liz Henry (:lizzard) (relman/hg->git project)]

Are we able to do anything for 66 here? I don't want to keep tracking it during the release, if not.

Comment 32

[Mike Taylor [:miketaylr]]

Not really no... it's up to Google to make a fix here. Probably not worth tracking.

Comment 33

[Pascal Chevrel:pascalc (PTO until April 26)]

(In reply to Mike Taylor [:miketaylr] from comment #32)

Not really no... it's up to Google to make a fix here. Probably not worth tracking.

Untracking and wontfix for 67 as this is not actionable on our side.

Comment 34

[Firefox Bug Husbandry Bot]

See bug 1547409. Moving webcompat whiteboard tags to keywords.

Comment 35

[Oana Arbuzov [:oanaarbuzov]]

I could not reproduce the issue on my side, the characters are not duplicated.
https://i.imgur.com/iPGoZyP.png

Tested with:
Browser / Version: Firefox Nightly 103.0a1 (2022-06-06)
Operating System: Ubuntu 20.04.2

Markus can you still reproduce the issue?

Comment 36

[Markus Popp]

This seems to work now.