863728 - Implement the replacement encoding

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[Henri Sivonen (:hsivonen)]

Implement a decoder for http://encoding.spec.whatwg.org/#replacement

Comment 1

[Henri Sivonen (:hsivonen)]

Attachment: Implement the encoding

Comment 2

[Henri Sivonen (:hsivonen)]

Attachment: Avoid the old encoding alias code

Comment 3

[Henri Sivonen (:hsivonen)]

Attachment: Tests

Comment 4

[Henri Sivonen (:hsivonen)]

https://tbpl.mozilla.org/?tree=Try&rev=7f3701ade31d

Comment 5

[Henri Sivonen (:hsivonen)]

Attachment: Tests, v2

Comment 6

[Henri Sivonen (:hsivonen)]

https://tbpl.mozilla.org/?tree=Try&rev=6f47ddb282ed

Comment 7

[Henri Sivonen (:hsivonen)]

https://tbpl.mozilla.org/?tree=Try&rev=6c63f246086f

Comment 8

[Henri Sivonen (:hsivonen)]

Attachment: The whole thing as one patch

Comment 9

[Henri Sivonen (:hsivonen)]

Comment on attachment 832142 [details] [diff] [review]
The whole thing as one patch

Some notes:
 * The changes from GetUnicodeDecoder to GetUnicodeDecoderRaw are there to avoid trips through the old alias table.

 * The bulk of the nsHtml5StreamParser changes are about deferring the commitment to a user-overridden encoding so that the replacement encoding can take precedence over the menu. Otherwise, socially engineered XSS would still work.

 * The replacement encoding doesn't inherit into iframes if you use script to inject an iframe into a replacement encoding document. This is mainly to keep the concept "doesn't inherit" and "ignores the menu" the same.

Comment 10

[Henri Sivonen (:hsivonen)]

Oh, and there's no actual dependency on bug 936439, because asserting stuff isn't really feasible before bug 805374 lands.

Comment 11

[Masatoshi Kimura [:emk]]

Comment on attachment 832142 [details] [diff] [review]
The whole thing as one patch

Review of attachment 832142 [details] [diff] [review]:
-----------------------------------------------------------------

Probably you will have to update moz2javaCharset in dom/plugins/base/nsPluginInstanceOwner.cpp.

::: dom/encoding/test/test_TextDecoder.js
@@ +352,4 @@
>      {encoding: "utf-16le", labels: ["utf-16", "utf-16le"]},
>      {encoding: "utf-16be", labels: ["utf-16be"]},
>      {encoding: "x-user-defined", labels: ["x-user-defined"]},
> +    {error: "TypeError", labels: ["x-windows-949", "\u0130SO-8859-1", "csiso2022kr", "iso-2022-kr", "iso-2022-cn-ext", "iso-2022-cn-ext", "replacement"]},

Why "iso-2022-cn-ext" is included twice? (One should be "iso-2022-cn".)

Comment 12

[Henri Sivonen (:hsivonen)]

(In reply to Masatoshi Kimura [:emk] from comment #11)
> Probably you will have to update moz2javaCharset in
> dom/plugins/base/nsPluginInstanceOwner.cpp.

Or delete the code, more likely: bug 942706.

> ::: dom/encoding/test/test_TextDecoder.js
> @@ +352,4 @@
> >      {encoding: "utf-16le", labels: ["utf-16", "utf-16le"]},
> >      {encoding: "utf-16be", labels: ["utf-16be"]},
> >      {encoding: "x-user-defined", labels: ["x-user-defined"]},
> > +    {error: "TypeError", labels: ["x-windows-949", "\u0130SO-8859-1", "csiso2022kr", "iso-2022-kr", "iso-2022-cn-ext", "iso-2022-cn-ext", "replacement"]},
> 
> Why "iso-2022-cn-ext" is included twice? (One should be "iso-2022-cn".)

Oops. Landed with that fixed:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3336625f130b

Thank you.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/3336625f130b