1441821 - cannot type kombuwa in sinhala wijesekara keyboard

Status: VERIFIED FIXED

(Core :: Widget: Win32, defect)

Description

[kpgtharaka]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0
Build ID: 20180206200532

Steps to reproduce:

just go to google.com 
entered ගෙදර in search box. 

in address bar and search box in firefox can type correctly. even in this textbox cannot type sinhala. (wijesekara)


OS : windows 10
issue happened on following versions.  
57.0.4
58.0.2


Actual results:

instead of showing ගෙදර it shows as දර (Kombuwa not apears)

‍ෙ and ග not displayed and one character backspaced.


Expected results:

when enter (f.or) it should show  ‍ගෙදර in input fields.

Comment 1

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to kpgtharaka from comment #0)
> just go to google.com 
> entered ගෙදර in search box. 
> 
> in address bar and search box in firefox can type correctly. even in this
> textbox cannot type sinhala. (wijesekara)

According to wikipedia <https://en.wikipedia.org/wiki/Sinhala_keyboard>, there are 2 Shinhara language keyboard layouts, Wijesekara layout and Windows Shinhara layout. When I check installed keyboard layouts on my Win10 environment, I see "Shinhara, touch keyboard layout" and "Shinhara - Wij 9, touch keyboard layout". The latter, i.e., Wij 9, is the Wijesekara layout? Otherwise, if you use 3rd party's keyboard layout, let me know whether I can get it from.

> Actual results:
> 
> instead of showing ගෙදර it shows as දර (Kombuwa not apears)

What's "Kmbuwa"?

> ‍ෙ and ග not displayed and one character backspaced.
> 
> 
> Expected results:
> 
> when enter (f.or) it should show  ‍ගෙදර in input fields.

Hmm, when I type "f.or" with both Shinhara keyboard layouts in my Win10 environment. I see both of them as:
"ෙ" "ග" "ද" "ර"
So, the first character and the second character are not merged. Should they be combined as "ගෙ"?

Comment 2

[kpgtharaka]

(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #1)
> (In reply to kpgtharaka from comment #0)
> > just go to google.com 
> > entered ගෙදර in search box. 
> > 
> > in address bar and search box in firefox can type correctly. even in this
> > textbox cannot type sinhala. (wijesekara)
> 
> According to wikipedia <https://en.wikipedia.org/wiki/Sinhala_keyboard>,
> there are 2 Shinhara language keyboard layouts, Wijesekara layout and
> Windows Shinhara layout. When I check installed keyboard layouts on my Win10
> environment, I see "Shinhara, touch keyboard layout" and "Shinhara - Wij 9,
> touch keyboard layout". The latter, i.e., Wij 9, is the Wijesekara layout?
> Otherwise, if you use 3rd party's keyboard layout, let me know whether I can
> get it from.

used "Sinhala touch keyboard layout", and tested wij 9. seems both are same layout using there. both having issue with only firefox. other applications working fine.

> 
> > Actual results:
> > 
> > instead of showing ගෙදර it shows as දර (Kombuwa not apears)
> 
> What's "Kmbuwa"?
called ‍in sinhala language for this  ‍"ෙ"
> 
> > ‍ෙ and ග not displayed and one character backspaced.
> > 
> > 
> > Expected results:
> > 
> > when enter (f.or) it should show  ‍ගෙදර in input fields.
> 
> Hmm, when I type "f.or" with both Shinhara keyboard layouts in my Win10
> environment. I see both of them as:
> "ෙ" "ග" "ද" "ර"
> So, the first character and the second character are not merged. Should they
> be combined as "ගෙ"?

it required Sinhala Tamil Kit for merge those
can obtain from here.
https://www.icta.lk/keyboard-drivers-etc/

BTW what is the firefox version you tested?

Comment 3

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to kpgtharaka from comment #2)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #1)
> > (In reply to kpgtharaka from comment #0)
> > > ‍ෙ and ග not displayed and one character backspaced.
> > > 
> > > 
> > > Expected results:
> > > 
> > > when enter (f.or) it should show  ‍ගෙදර in input fields.
> > 
> > Hmm, when I type "f.or" with both Shinhara keyboard layouts in my Win10
> > environment. I see both of them as:
> > "ෙ" "ග" "ද" "ර"
> > So, the first character and the second character are not merged. Should they
> > be combined as "ගෙ"?
> 
> it required Sinhala Tamil Kit for merge those
> can obtain from here.
> https://www.icta.lk/keyboard-drivers-etc/

So, did you report a bug with the 3rd party's keyboard driver? I'm still not sure if my environemnt's *behavior* is just different from your environment or, environment itself is so.

> BTW what is the firefox version you tested?

I'm using the latest Nightly build. Anyway, we haven't touched around keyboard event handling on Windows after releasing 57, IIRC.

Comment 4

[kpgtharaka]

(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #3)
> (In reply to kpgtharaka from comment #2)
> > (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #1)
> > > (In reply to kpgtharaka from comment #0)
> > > > ‍ෙ and ග not displayed and one character backspaced.
> > > > 
> > > > 
> > > > Expected results:
> > > > 
> > > > when enter (f.or) it should show  ‍ගෙදර in input fields.
> > > 
> > > Hmm, when I type "f.or" with both Shinhara keyboard layouts in my Win10
> > > environment. I see both of them as:
> > > "ෙ" "ග" "ද" "ර"
> > > So, the first character and the second character are not merged. Should they
> > > be combined as "ගෙ"?
> > 
> > it required Sinhala Tamil Kit for merge those
> > can obtain from here.
> > https://www.icta.lk/keyboard-drivers-etc/
> 
> So, did you report a bug with the 3rd party's keyboard driver? I'm still not
> sure if my environemnt's *behavior* is just different from your environment
> or, environment itself is so.
if it had a bug cannot type other places. even in firefox addressbar not having issue. it showing correctly.
problem is with text boxes within browsing area. all other places working fine. other browsers also having no issue 
that's why i open bug here

> 
> > BTW what is the firefox version you tested?
> 
> I'm using the latest Nightly build. Anyway, we haven't touched around
> keyboard event handling on Windows after releasing 57, IIRC.
tried nighty 60.0a1 same issue continuing. previously it was working fine. didn't change input methods or linking software.

Comment 5

[kpgtharaka]

today just tested following versions
56.0
55.0
both had the issue


later installed 52.0
it working fine without any issue.



so this happens before 57. but seems no one reported.

Comment 6

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to kpgtharaka from comment #4)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #3)
> > So, did you report a bug with the 3rd party's keyboard driver? I'm still not
> > sure if my environemnt's *behavior* is just different from your environment
> > or, environment itself is so.
> if it had a bug cannot type other places. even in firefox addressbar not
> having issue. it showing correctly.
> problem is with text boxes within browsing area. all other places working
> fine. other browsers also having no issue 
> that's why i open bug here

My question was, is this bug reproducible only when the 3rd party's keyboard driver? Or is this bug reproducible even only with Sinhala keyboard layouts of Windows 10?

Comment 7

[kpgtharaka]

all other browsers working without any issue. problem having with only firefox.

sinhala couldn't type without this driver.

Comment 8

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

I installed https://www.icta.lk/keyboard-drivers-etc/ into Win10 en-US on VMware.

Then, I reproduced only a few times in google.co.jp and google.com but I'm not sure if I see same website because Google checks IP address and may send different page to Japanese IP address users.

Anyway, the "keyboard-driver" is not simple keyboard layout driver. It's TIP (Text Input Processor) of TSF (Text Services Framework), that means IME implemented with TSF.  And unfortunately, it's not implemented with standard manner of IME. E.g., when I type "f." on https://w3c.github.io/uievents/tools/key-event-viewer.html , I see this odd result with Nightly (enabling 2 prefs to emulate almost same behavior with Chrome, though [1]):

18	keyup			0	190	190	ග	Period	-	-	'ග‌ෙ'
17	input			-	-	-	-	-	-	-	'ග‌ෙ'
16	compositionupdate	-	-	-	-	-	'ග‌ෙ'	-	'ග'
15	input			-	-	-	-	-	-	-	'ග'
14	compositionupdate	-	-	-	-	-	'ග'	-	''
13	compositionstart	-	-	-	-	-	''	-	''
12	input			-	-	-	-	-	-	-	''
11	compositionend	-	-	-	-	-	-	''	-	''
10	input			-	-	-	-	-	-	-	''
9	compositionupdate	-	-	-	-	-	''	-	'‍'
8	input			-	-	-	-	-	-	-	'‍'
7	compositionupdate	-	-	-	-	-	'‍'	-	'‍‌ෙ'
6	keyup			0	70F	70	ෙ	KeyF	-	-	'‍‌ෙ'
5	input			-	-	-	-	-	-	-	'‍‌ෙ'
4	compositionupdate	-	-	-	-	-	'‍‌ෙ'	-	'‍'
3	input			-	-	-	-	-	-	-	'‍'
2	compositionupdate	-	-	-	-	-	'‍'	-	''
1	compositionstart	-	-	-	-	-	''	-	''

Like this, there is no keydown events which should be dispatched before "compositionstart" and there are some redundant "compositionupdate" events. On the other hand, Chrome also fires almost same events. So, I guess that it might be Google's bug.

1: dom.keyboardevent.dispatch_during_composition and dom.keyboardevent.keypress.dispatch_non_printable_keys_only_system_group_in_content in about:config

Comment 9

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

As the result of the investigation, I have some questions now:

1. Do you reproduce this bug on the latest Nightly with setting "dom.keyboardevent.dispatch_during_composition" to true in about:config? (You don't need to restart Nightly after changing this pref.)
2. Do you reproduce this bug on the latest Nightly with setting "dom.keyboardevent.keypress.dispatch_non_printable_keys_only_system_group_in_content" to true in about:config? (You don't need to restart Nightly.)
3. Do you reproduce this bug on the latest Nightly with setting both "dom.keyboardevent.dispatch_during_composition" and "dom.keyboardevent.keypress.dispatch_non_printable_keys_only_system_group_in_content" to true if you reproduced this bug both #1 and #2.
4. Do you reproduce this bug on the latest Nightly with setting "intl.tsf.enable" to false? (You *need* to restart Firefox after changing this pref.)
5. Do you reproduce this bug if you switch UA string to Chrome on Windows with UserAgent switcher addon? <https://addons.mozilla.org/en-US/firefox/addon/user-agent-switcher-revived/?src=search>
6. Do you reproduce this bug if you switch UA string to Chrome on Windows and #1, #2 or #3?

Comment 10

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

FYI: Please use latest Nightly build because around "dom.keyboardevent.dispatch_during_composition" is now changed a lot in the last week.

Comment 11

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(And I've not seen any trouble on usual web pages yet.)

Comment 12

[kpgtharaka]

it's not we get google.co.jp. it is google.lk 

google ime

just installed nighty 61.0a1

here is the default result before change any on about:config

trying to type ගෙදර (f.or)

18	keyup	0	82R	82	'දර'
17	input	-	-	-	'දර'
16	keypress	3515 ර	0	3515	'ද'
15	keydown	0	0	0	'ද'
14	keyup	0	79O	79	'ද'
13	input	-	-	-	'ද'
12	keypress	3503 ද	0	3503	''
11	keydown	0	0	0	''
10	keyup	0	190	190	''
9	keypress	0 	8	8	''
8	keydown	0	8	8	''
7	input	-	-	-	''
6	keypress	0 	8	8	'‍'
5	keydown	0	8	8	'‍'
4	keyup	0	70F	70	'‍'
3	input	-	-	-	'‍'
2	keypress	8205 ‍	0	8205	''
1	keydown	0	0	0	''

tried switching UA string to Chrome
same result received

Comment 13

[kpgtharaka]

same driver but two results on two browsers

Firefox Nighty 


16	keyup	0	82R	82	ර	KeyR	0	✗	✗	-	-	-	'දර'
15	input	-	-	-	-	-	-	✗	✗	-	-	-	'දර'
14	keypress	3515 ර	0	3515	ර		0	✓	✗	-	-	-	'ද'
13	keydown	0	0	0	ර		0	✓	✗	-	-	-	'ද'
12	keyup	0	79O	79	ද	KeyO	0	✗	✗	-	-	-	'ද'
11	input	-	-	-	-	-	-	✗	✗	-	-	-	'ද'
10	keypress	3503 ද	0	3503	ද		0	✓	✗	-	-	-	''
9	keydown	0	0	0	ද		0	✓	✗	-	-	-	''
8	keyup	0	190	190	ග	Period	0	✗	✗	-	-	-	''
7	input	-	-	-	-	-	-	✗	✗	-	-	-	''
6	keypress	0 	8	8	Backspace		0	✓	✗	-	-	-	'‍'
5	keydown	0	8	8	Backspace		0	✓	✗	-	-	-	'‍'
4	keyup	0	70F	70	ෙ	KeyF	0	✗	✗	-	-	-	'‍'
3	input	-	-	-	-	-	-	✗	✗	-	-	-	'‍'
2	keypress	8205 ‍	0	8205	‍		0	✓	✗	-	-	-	''
1	keydown	0	0	0	‍		0	✓	✗	-	-	-	''


in chrome working without any issue
34	keyup	0	82R	82	ර	KeyR	0	✗	✗	-	-	-	'ගෙදර'
33	input	-	-	-	-	-	-	✗	✗	insertText	'ර'	-	'ගෙදර'
32	beforeinput	-	-	-	-	-	-	✗	✗	insertText	'ර'	-	'ගෙද'
31	keypress	3515 ර	3515	3515	ර		0	✗	✗	-	-	-	'ගෙද'
30	keydown	0	231	231	ර		0	✗	✗	-	-	-	'ගෙද'
29	keyup	0	79O	79	ද	KeyO	0	✗	✗	-	-	-	'ගෙද'
28	input	-	-	-	-	-	-	✗	✗	insertText	'ද'	-	'ගෙද'
27	beforeinput	-	-	-	-	-	-	✗	✗	insertText	'ද'	-	'ගෙ'
26	keypress	3503 ද	3503	3503	ද		0	✗	✗	-	-	-	'ගෙ'
25	keydown	0	231	231	ද		0	✗	✗	-	-	-	'ගෙ'
24	keyup	0	190	190	ග	Period	0	✗	✗	-	-	-	'ගෙ'
23	input	-	-	-	-	-	-	✗	✗	insertText	'ෙ'	-	'ගෙ'
22	beforeinput	-	-	-	-	-	-	✗	✗	insertText	'ෙ'	-	'ග'
21	keypress	3545 ෙ	3545	3545	ෙ		0	✗	✗	-	-	-	'ග'
20	keydown	0	231	231	ෙ		0	✓	✗	-	-	-	'ග'
19	input	-	-	-	-	-	-	✗	✗	insertText	'ග'	-	'ග'
18	beforeinput	-	-	-	-	-	-	✗	✗	insertText	'ග'	-	''
17	keypress	3484 ග	3484	3484	ග		0	✗	✗	-	-	-	''
16	keydown	0	231	231	ග		0	✓	✗	-	-	-	''
15	input	-	-	-	-	-	-	✗	✗	deleteContentBackward	'null'	-	''
14	beforeinput	-	-	-	-	-	-	✗	✗	deleteContentBackward	'null'	-	'‍'
13	keydown	0	8	8	Backspace		0	✓	✗	-	-	-	'‍'
12	input	-	-	-	-	-	-	✗	✗	deleteContentBackward	'null'	-	'‍'
11	beforeinput	-	-	-	-	-	-	✗	✗	deleteContentBackward	'null'	-	'‍ෙ'
10	keydown	0	8	8	Backspace		0	✗	✗	-	-	-	'‍ෙ'
9	keyup	0	70F	70	ෙ	KeyF	0	✗	✗	-	-	-	'‍ෙ'
8	input	-	-	-	-	-	-	✗	✗	insertText	'ෙ'	-	'‍ෙ'
7	beforeinput	-	-	-	-	-	-	✗	✗	insertText	'ෙ'	-	'‍'
6	keypress	3545 ෙ	3545	3545	ෙ		0	✗	✗	-	-	-	'‍'
5	keydown	0	231	231	ෙ		0	✓	✗	-	-	-	'‍'
4	input	-	-	-	-	-	-	✗	✗	insertText	'‍'	-	'‍'
3	beforeinput	-	-	-	-	-	-	✗	✗	insertText	'‍'	-	''
2	keypress	8205 ‍	8205	8205	‍		0	✗	✗	-	-	-	''
1	keydown	0	231	231	ද		0	✗	✗	-	-	-	''

Comment 14

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

That's really odd result. In your result, no composition events are included. That's really different result on my test environment. And you wrote "googole ime" which is never you commented.

Could you explain how to create same environment as yours? Which IME/keyboard layout/key utilies are necessary?

Comment 15

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to kpgtharaka from comment #12)
> tried switching UA string to Chrome same result received

Did you mean that you got same result on google.com even if you changed the UA string? Or did you tested only the keyboard event viewer? If you meant the latter, please test it in google.com (or google.lk) again and let me know the result?

Comment 16

[kpgtharaka]

(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #14)
> That's really odd result. In your result, no composition events are
> included. That's really different result on my test environment. And you
> wrote "googole ime" which is never you commented.

sorry for the distraction. ignore that google ime there. 

here is options which i got this result, compositionupdate checked there
https://ibb.co/jfs7hx

is there any option need to enable, tested changing followings but result was same

dom.keyboardevent.dispatch_during_composition 
dom.keyboardevent.keypress.dispatch_non_printable_keys_only_system_group_in_content

> 
> Could you explain how to create same environment as yours? Which
> IME/keyboard layout/key utilies are necessary?
only installed Sinhala Tamil Kit 

withing firefox browsing area getting same result. coudn't type ගෙ yet.  bug is still there

Comment 17

[kpgtharaka]

Firefox Nighty 62.0a1

Still this issue Continuing most of people left firefox and using other browsers due to this issue.

Comment 18

[Danishka]

I am still using Firefox 60.0.1 (64-bit) on Fedora (28) and no issue on Firefeox.
I just typed following these words on the bugzilla itself using Wijesekara input method.
I believe this is related to Windows.

ලෙදර 
මෙෙ
සෙකර
චෙබ
නෙප

Comment 19

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Yeah, I believe that it's caused by odd behavior of IME running on kpgtharaka's environment. However, I have not created same environment yet. And still no reply to my questions in comment 3 and comment 15.

I need:
1. exact steps to create same environment with Win10 for en-US.
2. exact steps to reproduce the symptom in simpler page like https://w3c.github.io/uievents/tools/key-event-viewer.html if it's possible (google's search field is too complicated web app).

Comment 20

[Danishka]

Tharaka, 

I have published a post regarding this to find if people still having issues.
https://www.facebook.com/danishka.navin/posts/10156612699842867

Could you please get those information in to this ticket please?

Comment 21

[Danishka]

one of my friend added two screenshots to my Facebook post and confirmed that its not reproducible in both Chrome and MS Office.

Comment 22

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Log file typying "f.or" with Sinhala - Wij9 and Sinhala Tamil keyboard driver

Okay, I reproduced this bug with following steps:

1. Install Win10 for en-US.
2. Add Sinhala language.
3. Add Sinhala - Wij 9 keyboard layout
4. Install Sinhala Tamil keyboard driver from https://www.icta.lk/keyboard-drivers-etc/
5. Install Firefox Nightly
6. Go to https://w3c.github.io/uievents/tools/key-event-viewer.html
7. Type "f.or"

Perhaps, if some East-Asian keyboard layout is installed, this bug is not reproducible since it installs full function of IME.

Comment 23

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Hmm, really crazy behavior...

When you type "f", 2 WM_KEYDOWN messages are sent with VK_PACKET.

First one inserts ZERO WIDTH JOINER (0x200D) with WM_CHAR.

The other inserts 'ෙ' (0x0DD9) with WM_CHAR.

Only the latter causes eKeyPress event.

Whn you type ".", 4 WM_KEYDOWN messages are sent.

First 2 messages are emulating Backspace key. So, trying to remove the inputted characters with "f" key press.

Then, next 2 messages are followed by 2 WM_CHAR messages 'ග' (0x0D9C) and 'ෙ' (0x0DD9).

Oddly, only the first WM_CHAR for ZERO WIDTH JOINER causes "keypress" event on web content.

Comment 24

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

I got the reasons, but the main reason is, Sinhala Tamil keyboard driver tells us wrong repeat count of each WM_KEYDOWN, WM_CHAR and WM_KEYUP. It sets repeat count to 1, but this should be 0. When we receive 1 or more, we may discard repeated key events due to performance reason.  If they set repeat count to 0 correctly, this bug won't occur.

Anyway, I'll create a hacky patch for existing this diver's bug.

Comment 25

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Hmm, it might be limitation of SendInput() API.
https://msdn.microsoft.com/en-us/library/windows/desktop/ms646310(v=vs.85).aspx
https://msdn.microsoft.com/en-us/library/windows/desktop/ms646271(v=vs.85).aspx

There is no document whether it can control 31th bit of lParam.

Comment 26

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3c4f6e9eb03acca23ffdd69e93163ede9272c190

Comment 27

[kpgtharaka]

(In reply to Danishka from comment #20)
> Tharaka, 
> 
> I have published a post regarding this to find if people still having issues.
> https://www.facebook.com/danishka.navin/posts/10156612699842867
> 
> Could you please get those information in to this ticket please?

from danishka's post we got following issues related to this bug

1. Priyantha Weerasinghe : having same issue when typing 'ෙ' (0x0DD9) and deletes both typed chars. not provide OS/FF version info 

2. Anoj Thabrew : reporting he couldn't type  'ප්‍ර' using 'm`' (simple m & tildi) letters. it appears as 'ප්'. he also having 'ෙ' (0x0DD9) letter deletion issue.  OS- Windows 10 ,64 bit, Firefox version 60.0.1 (64-bit) he said that it's similar to delete letters using backspace key

3. Thadika Walallawita : on Fedora after typing a word and then pressing spacebar or enter deletes last letter. (not only FF). he is having same isssue in Ubuntu (seems other issue/Not mentioned versions or IME used to type)

4. Thusitha Kumara : Not working properly, using 'ෙ' (0x0DD9) deletes the letter typed

Comment 28

[kpgtharaka]

(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #22)
> Created attachment 8983674 [details]
> Log file typying "f.or" with Sinhala - Wij9 and Sinhala Tamil keyboard driver
> 
> Okay, I reproduced this bug with following steps:
> 
> 1. Install Win10 for en-US.
> 2. Add Sinhala language.
> 3. Add Sinhala - Wij 9 keyboard layout
> 4. Install Sinhala Tamil keyboard driver from
> https://www.icta.lk/keyboard-drivers-etc/
> 5. Install Firefox Nightly
> 6. Go to https://w3c.github.io/uievents/tools/key-event-viewer.html
> 7. Type "f.or"
> 
> Perhaps, if some East-Asian keyboard layout is installed, this bug is not
> reproducible since it installs full function of IME.

Thanks finally you were able to reproduce it. 
The thing is this not happens until FF 56.0. it occurred after FF 57.0 hope you can test it too.

Comment 29

[kpgtharaka]

(In reply to kpgtharaka from comment #28)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #22)
> > Created attachment 8983674 [details]
> > Log file typying "f.or" with Sinhala - Wij9 and Sinhala Tamil keyboard driver
> > 
> > Okay, I reproduced this bug with following steps:
> > 
> > 1. Install Win10 for en-US.
> > 2. Add Sinhala language.
> > 3. Add Sinhala - Wij 9 keyboard layout
> > 4. Install Sinhala Tamil keyboard driver from
> > https://www.icta.lk/keyboard-drivers-etc/
> > 5. Install Firefox Nightly
> > 6. Go to https://w3c.github.io/uievents/tools/key-event-viewer.html
> > 7. Type "f.or"
> > 
> > Perhaps, if some East-Asian keyboard layout is installed, this bug is not
> > reproducible since it installs full function of IME.
> 
> Thanks finally you were able to reproduce it. 
> The thing is this not happens until FF 56.0. it occurred after FF 57.0 hope
> you can test it too.

small correction. wrong versions mentioned
52.0 was working fine. both 56.0 and 57.0 having issue

Comment 30

[kpgtharaka]

just installed Firefox Versions 55.0, 54.0, 53.0 all having the issue

But Firefox Version 52.0 working fine without any issue.

so problem starts with Firefox Version 53.0

hope this info also helpful to fix this issue

Comment 31

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Could you test these builds?:
x86: https://queue.taskcluster.net/v1/task/fuMcsvpoQ2SSKlHeUw5SqA/runs/0/artifacts/public/build/target.zip
x86_64: https://queue.taskcluster.net/v1/task/PcTRAMdJTuqmyusAo_nTdA/runs/0/artifacts/public/build/target.zip

Comment 32

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

This is a regression of bug 1330252, but it depends on if e10s is enabled on your environment whether you can reproduce this with 53 ~ 56.

Comment 33

[kpgtharaka]

Tested both builds.

sinhala typing working fine on both.


able to type x32
ගෙදර
මෛ
ප්‍රකාර
වර්‍ගය

on x85_64
ගෙදර
මෛ
ප්‍රකාර
වර්‍ගය

Comment 34

[kpgtharaka]

(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #32)
> This is a regression of bug 1330252, but it depends on if e10s is enabled on
> your environment whether you can reproduce this with 53 ~ 56.

while i submitted previous comment seems your blocks 1330252 was changed

Comment 35

[Danishka]

I have asked few people to test.
Lets wait for their confirmation

Comment 36

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to Danishka from comment #35)
> I have asked few people to test.
> Lets wait for their confirmation

Thanks, but please wait a moment. Safer build is coming.

(In reply to kpgtharaka from comment #34)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #32)
> > This is a regression of bug 1330252, but it depends on if e10s is enabled on
> > your environment whether you can reproduce this with 53 ~ 56.
> 
> while i submitted previous comment seems your blocks 1330252 was changed

Thank you, and don't mind.

Comment 37

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c2c31da8b4fefca176e6cb036ab12d4cee0b3e73

Comment 38

[Danishka]

@Masayuki,

Seems previous links to 32bit and 64bit build are no longer available.
Can you share the correct link pls?

Comment 39

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Here are new builds (with safer approach to fix this bug):
x86: https://queue.taskcluster.net/v1/task/Wrd8hIY-SMuRCvQpDP8Bbg/runs/0/artifacts/public/build/target.zip
x86_64: https://queue.taskcluster.net/v1/task/YNe2m9CJQhKbqewW-vJ4jQ/runs/0/artifacts/public/build/target.zip

Comment 40

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Attachment: Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press

Currently, TabChild discards eKeyDown ane eKeyPress events which are marked as
"repeated" and were dispatched after the latest eKeyDown event comes into the
process.  However, keyboard layout utils may generate native key events
as "repeated" even if each native key is important to input proper text.

So, TabChild shouldn't decide if coming keyboard event is skippable only with
mIsRepeat.  For solving this issue, this patch adds
mMaybeSkippableInRemoteProcess to WidgetKeyboardEvent and makes
TabChild::SkipRepeatedKeyEvent() check
WidgetKeyboardEvent::CanSkipInRemoteProcess() instead.

On Windows, there are two ways to generate keyboard input messages.  One is
using SendMessage() or PostMessage().  The other is SendInput() API.  In both
ways, utils can make their input as repeated key messages.

The former case must be safe for this issue since such utils need to set 31st
bit of lParam to 1 explicitly.

On the other hand, in the latter case, the utils probably need to append
KEYEVENTF_KEYUP into KEYBDINPUT::dwFlags.  Otherwise, only first call is
treated as non-repeated event.

So, when given message does not came from physical key operation, NativeKey
should set WidgetKeyboardEvent::mMaybeSkippableInRemoteProcess to false
even if WidgetKeyboardEvent::mIsRepeat is true.

Review commit: https://reviewboard.mozilla.org/r/249892/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/249892/

Comment 41

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

I'd like Makoto-san to review widget/windows and smaug to review the other part.

kpgtharaka and Danishka:

I think that we should land the patches as soon as possible and if it'd be allowed, we should fix current beta and 60 ESR since this is serious regression for Sinhala and Tamil language users. So, if you have time to check the new test builds in comment 39, please check them and if you see something wrong behavior, let me know.

Comment 42

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

This is not a recent regression, since 53 though (and reproduced only with e10s), this is too serious regression for Sinhala and Tamil language users. So, I believe that we should uplift the patches as soon as possible.

Comment 43

[Danishka]

(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #41)
> I'd like Makoto-san to review widget/windows and smaug to review the other
> part.
> 
> kpgtharaka and Danishka:
> 
> I think that we should land the patches as soon as possible and if it'd be
> allowed, we should fix current beta and 60 ESR since this is serious
> regression for Sinhala and Tamil language users. So, if you have time to
> check the new test builds in comment 39, please check them and if you see
> something wrong behavior, let me know.

Masayuki,
Tested several people and so far no issue with the latest build.
I highly appreciated your support on fixing this issue and this issue cause many Windows users give up Firefox as they could not type Sinhala as they were expected.

Lets get the patch available on next release asap.

Comment 44

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to Danishka from comment #43)
> Masayuki,
> Tested several people and so far no issue with the latest build.
> I highly appreciated your support on fixing this issue and this issue cause
> many Windows users give up Firefox as they could not type Sinhala as they
> were expected.

Thank you. And sorry for the delay to fix this bug.

Perhaps, we should report the bug of SinhalaTamil IME's developer. I'll look for a contact...

Comment 45

[Ryan VanderMeulen [:RyanVM]]

We're about to enter the last week of the Beta cycle for Fx61 and this patch looks pretty scary for a late-cycle uplift, especially for a regression that's been there since 53. Not tracking for 61, but I won't slam the door shut on an uplift if you can convince me that this isn't as risky as it appears.

Comment 46

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8984345 [details]
Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press

https://reviewboard.mozilla.org/r/249892/#review256628

Interesting bug. Sinhala writing system looks beautiful.

::: commit-message-ea21b:3
(Diff revision 1)
> +Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press r?m_kato, smaug
> +
> +Currently, TabChild discards eKeyDown ane eKeyPress events which are marked as

S/ane/and/

::: widget/TextEvents.h:407
(Diff revision 1)
> +
> +  bool CanSkipInRemoteProcess() const
> +  {
> +    // If this is a repeat event (i.e., generated by auto-repeat feature of
> +    // the platform), remote process may skip to handle it since delayed
> +    // key events may operate something unexpectedly.  However, if it's

Not sure what this sentence is trying to capture, but skipping events was added because or performance. Flooding main event queue with repeated key events was making page downs on google presentation app slow.

So, perhaps change to something like
"..., remove process may skip to handle it because of performances reasons."

::: widget/windows/KeyboardLayout.h:432
(Diff revision 1)
>    // any modifier keys.  Otherwise, false.
>    // Please note that the event may not cause any text input even if this
>    // is true.  E.g., it might be dead key state or Ctrl key may be pressed.
> -  bool    mIsPrintableKey;
> +  bool mIsPrintableKey;
> +  // mIsSkippableInRemoteProcess is false if the key event shouldn't be
> +  // skipped to handle in the remote process even if it's too old event.

just drop "to handle"

::: widget/windows/KeyboardLayout.cpp:1292
(Diff revision 1)
> +    case WM_SYSCHAR:
> +    case WM_DEADCHAR:
> +    case WM_SYSDEADCHAR:
> +      // However, some keyboard layouts may send some keyboard messages with
> +      // activating the bit.  If we dispatch repeated keyboard events, they
> +      // may be ignored by TabChil due to performance reason.  So, we need

TabChild

Comment 47

[Makoto Kato [:m_kato]]

Comment on attachment 8984345 [details]
Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press

https://reviewboard.mozilla.org/r/249892/#review256796

Comment 48

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Comment on attachment 8984345 [details]
Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/249892/diff/1-2/

Comment 49

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3049615be4fe
NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press r=m_kato,smaug

Comment 50

[kpgtharaka]

(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #41)
> I'd like Makoto-san to review widget/windows and smaug to review the other
> part.
> 
> kpgtharaka and Danishka:
> 
> I think that we should land the patches as soon as possible and if it'd be
> allowed, we should fix current beta and 60 ESR since this is serious
> regression for Sinhala and Tamil language users. So, if you have time to
> check the new test builds in comment 39, please check them and if you see
> something wrong behavior, let me know.

Thank you very much for identify there was a issue and resolve it. 
i also asked several people to test your build and result was good. they were able to type same as previous.  

(In reply to Ryan VanderMeulen [:RyanVM] from comment #45)
> We're about to enter the last week of the Beta cycle for Fx61 and this patch
> looks pretty scary for a late-cycle uplift, especially for a regression
> that's been there since 53. Not tracking for 61, but I won't slam the door
> shut on an uplift if you can convince me that this isn't as risky as it
> appears.

first of all i though this error was only for me. later while discussing with friend most of them leave firefox due to that they couldn't type in their native language. unfortunately no one identify this is a issue with firefox. so it came here until i report this.

Comment 51

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=924dac44d7c8671aa6b300a0296644ac7e448632

Comment 52

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to kpgtharaka from comment #50)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #41)
> > I'd like Makoto-san to review widget/windows and smaug to review the other
> > part.
> > 
> > kpgtharaka and Danishka:
> > 
> > I think that we should land the patches as soon as possible and if it'd be
> > allowed, we should fix current beta and 60 ESR since this is serious
> > regression for Sinhala and Tamil language users. So, if you have time to
> > check the new test builds in comment 39, please check them and if you see
> > something wrong behavior, let me know.
> 
> Thank you very much for identify there was a issue and resolve it. 
> i also asked several people to test your build and result was good. they
> were able to type same as previous.  

Thank you. Unfortunately, even with my patch, KeyboardEvent.repeat value is odd when a Unicode character is inputted or a Backspace key press is emulated by Sinhala Tamil IME. I contacted this bug to ICTA with explaining possible bug cause and a way to fix. It might be updated if they work on that.

> (In reply to Ryan VanderMeulen [:RyanVM] from comment #45)
> > We're about to enter the last week of the Beta cycle for Fx61 and this patch
> > looks pretty scary for a late-cycle uplift, especially for a regression
> > that's been there since 53. Not tracking for 61, but I won't slam the door
> > shut on an uplift if you can convince me that this isn't as risky as it
> > appears.
> 
> first of all i though this error was only for me. later while discussing
> with friend most of them leave firefox due to that they couldn't type in
> their native language. unfortunately no one identify this is a issue with
> firefox. so it came here until i report this.

Hmm, that's sad. But Firefox should always be an alternative browser for any language users. We should keep fixing too serious i18n bugs ASAP.

Comment 53

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/3049615be4fe

Comment 54

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Comment on attachment 8984345 [details]
Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press

Approval Request Comment
[Feature/Bug causing the regression]:
Bug 1330252 (since Fx 53 if e10s is enabled, so, maybe a lot of users may see at 57 first).

[User impact if declined]:
Sinhala and Tamil languages users cannot type text in web content as expected. So, really serious regression.

[Is this code covered by automated tests?]:
Unfortunately, no. Since we don't have testing API to call SendInput() Win32 API from JS.

[Has the fix been verified in Nightly?]:
Not yet, but the tryserver builds are tested by the reporter.
[Needs manual test from QE? If yes, steps to reproduce]:
Yes.

1. Install Sinhala language and Sinhala Wij 9 keyboard layout on Win10 (Note that if you have already installed East-Asian keyboard layout, IME module may change the following driver's behavior and you cannot reproduce this bug).
2. Install Sinhala Tamil keyboard driver from https://www.icta.lk/keyboard-drivers-etc/ (So, I strongly recommend that you should test on fresh Win10 environment on VM).
3. Load https://w3c.github.io/uievents/tools/key-event-viewer.html
4. Type "f.or" as en-US layout into the field with Sinhala Wij 9 layout.
5. You should see "ගෙදර" in the field. And you should see 28 events including "Backspace" key events (they are generated by the driver).

And the patch touches initializing KeyboardLayout.repeat value initialization on Windows. So,

1. Select en-US keyboard layout.
2. Load https://w3c.github.io/uievents/tools/key-event-viewer.html
3. Press "a" key for a moment.
4. Then, first keydown and keypress shouldn't be marked as "repeat" but following keydown and keypress events should be marked as "repeat". keyup should not marked as "repeat".

[List of other uplifts needed for the feature/fix]:
Nothing.

[Is the change risky?]:
Not so risky, but the patch is not small, unfortunately.

[Why is the change risky/not risky?]:
Needed to move IsRepeat() method to mIsRepeat member which is initialized in InitWithAppCommand() and InitWithKeyOrChar().  This shouldn't be changed the behavior actually since mIsRepeat will be exposed to web via KeyboardEvent.repeat.

However, the main change of this patch is, even when mIsRepeat is true, TabChild should check new flag, it indicates whether the key event is skippable in remote process when the event is too old.

This new flag is NOT exposed to web and this is initialized only with the new method, NativeKey::InitIsSkippableForKeyOrChar(). This method makes this patch bigger, but this does NOT touch any existing behavior.

Therefore, if there were some regressions, the new flag in WidgetKeyboardEvent is false (its default is true). However, even in such case, we'd see only performance issue when user pressing a key long time. (So, I believe that comparing with this symptom, we should uplift this patch.)

[String changes made/needed]:
Nothing.

Comment 55

[Timea Cernea [:tbabos][inactive]]

I can confirm that the issue no longer occurs on Windows 10 x64, Ubuntu 16.04 and Mac OS 10.12.

It should be known that I verified the issue on Windows 10 using the above mentioned driver while on Unbuntu and Mac I changed the keyboard layout via the system settings. 
On all the platforms the characters were displayed as "ගෙදර", however please note that on Windows 10 (using the driver) I only got 26 events(check the attached screenshot), can this be considered as a problem?

Furthermore, I also checked KeyboardLayout.repeat which worked as described in the STR on Windows.

Comment 56

[Timea Cernea [:tbabos][inactive]]

Attachment: Screenshot for qe verify

Comment 57

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to Timea Babos from comment #55)
> I can confirm that the issue no longer occurs on Windows 10 x64, Ubuntu
> 16.04 and Mac OS 10.12.

Thank you very much!

> On all the platforms the characters were displayed as "ගෙදර", however please
> note that on Windows 10 (using the driver) I only got 26 events(check the
> attached screenshot), can this be considered as a problem?

It's difficult to say. We just exposed native event behavior with DOM events. Oddly, the keyboard driver, Sinhala Tamil IME, generates the odd/redundant keyboard events to native applications including us. If we'd treat the odd native key events as a part of IME composition, that may be reported as more natural behavior for web apps (from point of the view of DOM Events spec).

However, doing that may cause regressions since Chrome behaves exactly same as our behavior except beforeinput event. Edge's behavior looks nicer since it handles as IME composition, but perhaps, UWP application receives different events than desktop applications such as Firefox and Chrome.

So, the answer is no at least for now since taking incompatible behavior with Chrome may cause making some web apps incompatible especially on web apps whose main target is Sinhala or Tamil language users.

> Furthermore, I also checked KeyboardLayout.repeat which worked as described
> in the STR on Windows.

Yeah, I reported their bugs to the vendor, but I've not receive any reply from them.

Comment 58

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8984345 [details]
Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press

While I understand that this causes pretty bad usability issues for affected users, it's really late to be taking this patch at this point (we'd basically be landing it directly into a release candidate build). I think we should let this bake on 62 so it can get more testing on Beta before we ship it.

Comment 59

[Timea Cernea [:tbabos][inactive]]

Based on Comment 55 and Comment 57, I will mark this issue as Verified - Fixed.

Comment 60

[Danishka]

All,

I highly appreciated your support on fixing this bug which highly impact on Windows users in our community.

Comment 61

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=65302a4e891884fed62e196a05593119d4f5ee1d

Comment 62

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

Comment on attachment 8984345 [details]
Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
This bug makes Sinhala and Tamil language users cannot type their chracters on Windows. Regression of bug 1330252, starting from 53 only with e10s.

User impact if declined:
Sinhala and Tamil language users cannot type their chracters on Windows if they use SinhalaTamil IME mentioned in comment 2.

Fix Landed on Version:
62.

Risk to taking this patch (and alternatives if risky):
Must be low.

This patch just adds additional flag to repeated keydown event which indicates whether remote process can discard too old keydown event or not. So, this is related to KeyboardEvent.repeat, but the value is never changed.

String or UUID changes made by this patch:
Nothing.

See also comment 54 for the detail.

Comment 63

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8984345 [details]
Bug 1441821 - NativeKey shouldn't mark eKeyDown and eKeyPress as "skippable in remote process" if message is not caused by physical key press

This has been baking without any known regressions for awhile now. Let's take this for ESR 60.2 and make Firefox usable for Sinhala and Tamil language users again.

Comment 64

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/e290aa06c191

Comment 66

[Timea Cernea [:tbabos][inactive]]

I can confirm that the issue no longer occurs on Windows 10 x64, Ubuntu 16.04 and Mac OS 10.13.

Verified the issue on ESR 60, using the build from: https://tools.taskcluster.net/index/gecko.v2.mozilla-esr60.latest.firefox.