Closed Bug 1288298 Opened 6 years ago Closed 5 years ago

Fix Unicode encoding mismatch between JS-Plugins and the PPAPI Plug-in.

Categories

(Core Graveyard :: Plug-ins, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: brsun, Unassigned)

References

Details

Attachments

(1 file)

PPAPI uses UTF-8 as the encoding for Unicode strings (i.e. str_t), but JavaScript uses UCS-2/UTF-16 instead. We need to figure out how to convert the Unicode schemes back and forth properly in our design.
There is a dedicated Web API: https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder

But there is another one for the lazy bones, which encodes the JS string into the old and fake "binary string":

var bin_str = unescape(encodeURI(str));
We do use TextEncoder API in jsplugin, but as I know this bug will dig into Python and C++ generator code to make sure encoding in lower layer's works well.
... but maybe jsplugin need to be modified too. Thanks for the info. :)
Priority: -- → P3
Attachment #8794098 - Flags: review?(peterv)
I think I forget to mention the changes of this pull-request on this bug.

In summary, this pull-request is used to let JavaScript world uses its native strings without caring about the encoding transformation to and from |str_t| of pepper interfaces.

This pull-request includes:
 - Using only printable characters of ASCII in JSON strings while converting from UTF-8 characters.
 - A new class (i.e. |UnicodeString|) and a new modifier (i.e. |UnicodeEncoding|) to handle transcoding between UTF-8 encoded strings and printable-characters-only JSON strings.
 - A new MAKE target (i.e. |test-unicode|) for testing the correctness of transcoding.
 - A new environment variable (i.e. |STRICT|) for compiling codes with verbose-warnings and warnings-as-errors.
 - Add one workaround (i.e. |PPAPIUtility.getStringLength(...)|) in JavaScript world to calculate the string-length for pepper interfaces.
 - Add comments in runtime for each customized parameter.
 - Corresponding changes of RPC.
>  - A new class (i.e. |UnicodeString|) and a new modifier (i.e.
> |UnicodeEncoding|) to handle transcoding between UTF-8 encoded strings and
> printable-characters-only JSON strings.

s/modifier/manipulator
Comment on attachment 8794098 [details] [review]
https://github.com/mozilla/flapper.js/pull/62

According to the discussion offline, we would do transcoding stuffs in Gecko instead of doing that in RPC. If we really encounter performance issues, we would breakdown the issues and have further studies by then.
Attachment #8794098 - Flags: review?(peterv)
Un-assign myself because this bug would be handled by Peter's patch.
Assignee: brsun → nobody
Group: mozilla-employee-confidential
Can't reproduce this issue anymore.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
Resolution: WONTFIX → WORKSFORME
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.