496630 - Input if Unicode only scripts to plugins not possible on Windows

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, defect)

Description

[Jeff Mott]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 (.NET CLR 3.5.30729)

Flash Player hosted in firefox recieves '?' (uparam = 63) instead of useful code points in WM_CHAR events when input is Hindi.

Reproducible: Always

Steps to Reproduce:
1. Load foo.swf
2. Try to type Hindi in text field

Actual Results:  
Question marks:  ????

Expected Results:  
Hindi text:  पगल्ग

For other input methods WM_CHAR is giving us the right Unicode values, but not for Hindi for some reason.  I've also tested this with the 3.1 beta 3 player, same results.

Comment 1

[Jeff Mott]

Attachment: test file

Comment 2

[Jeff Mott]

Please note that this affects the input of Unicode only scripts on all platforms.  If there is no codepage for the script, the browser can't deliver valid text input data to the plugin.  Arabic on Mac has the same issue.

Comment 3

[Jeff Mott]

When the browser forwards a WM_CHAR event to us, for most scripts the WPARAM of the event is the Unicode code point.  But for Unicode only scripts, scripts without a code page (such as Hindi), something is going wrong inside of Netscape and they are sending us the Latin key code for the key pressed instead.  Something similar must be happening on Mac and Linux.

Here is a likely list of the scripts which do not map easily to code pages 
(may be because there is none): Ethiopic, Tifinagh, Yi, Cherokee, 
Canadian Syllabics, Deseret, Shavian, Vai, Tagalog, Hanunoo, Buhid, 
Tagbanwa, Khmer, Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, 
Telugu, Kannada, Malayalam, Thaana, Tibetan.

Comment 4

[chris hofmann]

does anyone know what the fix for this might look like?

Comment 5

[Jeff Mott]

Here's some more info:

These are the windows involved when the player gets input from Firefox:
 
0x00040132 "Mozilla Firefox" {MozillaUIWindowClass} (-8,-8),(1608,1178) -> "0" (Unicode) path="firefox.exe"
  0x00060760 "" {MozillaWindowClass} (0,20),(1600,1170) -> "0" (Unicode) path="firefox.exe"
    0x00090686 "" {MozillaWindowClass} (0,157),(1600,1148) -> "0" (Unicode) path="firefox.exe"
      0x00380640 "" {MozillaWindowClass} (0,157),(1600,1148) -> "0" (Unicode) path="firefox.exe"
        0x000C0652 "" {MozillaContentWindowClass} (0,157),(1600,1148) -> "0" (Unicode) path="firefox.exe"
          0x001907A0 "" {MozillaWindowClass} (0,157),(1600,1148) -> "0" (Unicode) path="firefox.exe"
            0x0009068E "" {MozillaWindowClass} (0,157),(1600,1148) -> "0" (Unicode) path="firefox.exe"
              0x000E0654 "" {MozillaWindowClass} (8,185),(808,665) -> "0" (ANSI) path="firefox.exe"
 
The ANSI window is the root cause of the trouble, and it is the one we hook in NPP_SetWindow.  That is the window Mozilla creates for plugins to draw into.

Change it to Unicode and the bug should go away.  We've experimented with adding an extra Unicode window to the stack and hooking to that just as a test, and it works for keyboard input, but we'd prefer not to ship something like that.

Comment 6

[Jeff Mott]

That's for Windows of course, not clear what the cause is on Mac.

Comment 7

[Sorin Sbarnea]

I should add that this bug reproduces for Romanian language for two characters: ș and ț. When use is trying to enter them it will see ?, pasting them works but this is not quite a solution.

Comment 8

[Masatoshi Kimura [:emk]]

We are using ANSI Window to avoid your ancient plug-ins crashing (See bug 179822).
We could use Unicode Window if you agree to blocklist those outdated plug-ins.
Please respond to bug 436348 :-)

Comment 9

[Jeff Mott]

I've asked our security expert to take a look, thanks.

Comment 10

[Jeff Mott]

As for 179822, we no longer support those old players.  Please change the window over to Unicode at your earliest opportunity.  And hopefully there's some equivalent on the Mac side?

Comment 11

[Masatoshi Kimura [:emk]]

I've removed ANSI window hack locally, but I couldn't still input Indic characters.
Flash Player seems to use SetWindowLongA/DefWindowProcA to subclass our window. SetWindowLongA changes window's Unicodeness to ANSI. So plug-in windows will become ANSI even if we fixed our side.
Jeff, are you planning to release a full Unicode version of Flash Player? If so, can I obtain a test build?
And sorry, I don't know about Mac.

Comment 12

[Jeff Mott]

We have a public beta of the Unicode player 10.1 coming out any day now, best bet is to grab that.  I'll try to remember to ping here when it's available.

Comment 13

[Jeff Mott]

Public beta available now, here:  http://labs.adobe.com/technologies/flashplayer10/

Comment 14

[Jeff Mott]

The inline IME in player 10.1 supports reconversion.  The Ansi window appears to be stepping on the reconversion data as well.

Comment 15

[Masatoshi Kimura [:emk]]

I could input Hindi using latest Minefield with Flash Player 10.1 beta 2 because our plug-in module is built with "-DUNICODE -D_UNICODE" options after Electrolysis landing.
http://mxr.mozilla.org/mozilla-central/source/modules/plugin/base/src/Makefile.in#114
http://mxr.mozilla.org/mozilla-central/source/ipc/chromium/chromium-config.mk#78
I'll keep this bug open because this bug is nominating for wanted1.9.2.
Electrolysis will not be landed on 192 branch, so we need to make a branch only patch if we're going to fix this bug for Firefox 3.6.

Comment 16

[Masatoshi Kimura [:emk]]

Attachment: patch (branch only)

Comment 17

[Masatoshi Kimura [:emk]]

Comment on attachment 418635 [details] [diff] [review]
patch (branch only)

This patch is virtually baked on trunk because Electrolysis landing has the same effect as this.

Comment 18

[Benjamin Smedberg]

Comment on attachment 418635 [details] [diff] [review]
patch (branch only)

Do we know what exactly is causing the fix? Which API are we calling the "A" version currently and would be calling the "W" version with this patch? I'm somewhat concerned about this without some additional context.

Comment 19

[Masatoshi Kimura [:emk]]

I can explain, but I will no longer proceed further because Lorentz project will fix this anyway.

Comment 20

[Masatoshi Kimura [:emk]]

Jeff, please file separate bugs for other platforms.

Comment 21

[Reed Loden [:reed]]

Uh, this is not a dupe of bug 536188.

Comment 22

[Masatoshi Kimura [:emk]]

This bug is already fixed on trunk and will be fixed on 192 branch when Lorentz branch is merged to 192 branch. How should I set the status of this bug?

Comment 23

[Benjamin Smedberg]

I would like to know *how* lorentz fixed the bug. Are you sure it was adding -DUNICODE in modules/plugin, or was it adding -DUNICODE in widget/src/windows? There is no guarantee that when we backport to 1.9.2 that we'll have the same situation, and I don't like bugs without testcases or understanding. Please explain how it works, if only to soothe my fears about unintended side effects.

Comment 24

[Masatoshi Kimura [:emk]]

(In reply to comment #23)
It was adding -DUNICODE in modules/plugin.
It changes the definition of SubclassWindow from SubclassWindowA to SubclassWindowW.
http://mxr.mozilla.org/mozilla-central/source/modules/plugin/base/src/nsPluginNativeWindowWin.cpp#549
We subclass plug-in windows here after Flash Player subclasses them. So the Unicodeness of plug-in windows is affected by our -DUNICODE option.

Comment 25

[Benjamin Smedberg]

Is it possible to write a test that doesn't require user interaction? I think we should, entirely apart from whether -DUNICODE is in force, use the explicit SubclassWindowW function (if we think that's safe).

Comment 26

[Benjamin Smedberg]

Comment on attachment 418635 [details] [diff] [review]
patch (branch only)

The best/smallest patch here would be to explicitly use SubclassWindowW, which is what we should do on both trunk and branch.

Comment 27

[Masatoshi Kimura [:emk]]

Sorry, I've forgotten to clear the review request.

Comment 28

[Masatoshi Kimura [:emk]]

Fixed on Minefield and Lorentz.

Comment 29

[Al Billings [:abillings - ex-MoCo]]

(In reply to comment #0)
> Steps to Reproduce:
> 1. Load foo.swf
> 2. Try to type Hindi in text field
> 
> Actual Results:  
> Question marks:  ????
> 
> Expected Results:  
> Hindi text:  पगल्ग


If I paste the Hindi text in above with the attached test file on XP, I don't see an issue. I assume that I have to use the IME to input text and that pasting works around the bug (as it does with other IME issues). Can someone give me some specific steps using the IME to reproduce this issue?

Comment 30

[Jeff Mott]

Load the test file (attached) in a shipping version of Firefox, using the 10.1 beta player.  Set your input mode to Hindi.  Put the focus in the input field, hit the 'm' key.  You should see 'स' in the text field, but instead you get '?'.  In theory if you do the same with the latest minefield it works - I haven't had a chance to try yet.

Comment 31

[Al Billings [:abillings - ex-MoCo]]

Thanks.

Interesting. When I do the above steps, I see the same results in 1.9.2.3 (the released version) and the nightly 1.9.2 build on Windows XP.

I load the testfile, place focus in the text input area within it, change my input method to Hindi with the language bar, and hit m there. In both instances, I get a '?' character. If I place focus in the address bar or the search bar and enter text, it shows up in Hindi, matching the character above ('स').

Either I'm doing something stupid (highly possible) or this isn't fixed in Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.5pre) Gecko/20100426 Namoroka/3.6.5pre (.NET CLR 3.5.30729).

Comment 32

[Masatoshi Kimura [:emk]]

(In reply to comment #30)
Please try with Firefox 3.6.4 beta.
http://www.mozilla.com/en-US/firefox/all-beta
Shipping versions of Firefox don't support Unicode plug-ins yet.

(In reply to comment #31)
Are you using Flash Player 10.1 beta?
http://labs.adobe.com/technologies/flashplayer10/

At present, Hindi input is supported only by a combination of Firefox beta and Flash Player beta.

Comment 33

[Al Billings [:abillings - ex-MoCo]]

I didn't try it in the beta because it looked like it was fixed after the beta was built. All of the Lorrentz fixes are *in* the build I tested with, Masatoshi. They are in the main 1.9.2 branch. Please examine my build ID. If it isn't fixed there, it isn't fixed in the Firefox 3.6.4 beta. 

I'm using the released Flash 10 because there is no note here that this only works with 10.1. 

So you're saying that this will not work with Flash 10 at all?

Comment 34

[Jeff Mott]

Correct, this will not work with Flash 10, only 10.1 and later.

Comment 35

[Al Billings [:abillings - ex-MoCo]]

Using the 10.1 test flash, I have verified this fixed for 1.9.2 in the current 1.9.2 nightly (https://bugzilla.mozilla.org/attachment.cgi?id=381838) and the 3.6.4 beta 1 (Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4 ( .NET CLR 3.5.30729)).