Remove JIS X 0212 support from EUC-JP encoder

RESOLVED FIXED in mozilla5

Status

()

RESOLVED FIXED
8 years ago
8 years ago

People

(Reporter: emk, Assigned: emk)

Tracking

({html5})

Trunk
mozilla5
html5
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(status2.0 ?)

Details

(URL)

Attachments

(1 attachment)

(Assignee)

Description

8 years ago
Recently, CP51932 has added as a replacement encoding for EUC-JP.
Roughlt speaking, CP51932 is an Microsoft variant of EUC-JP but it doesn't support JIS X 0212. It had been a severe interoperability problem for a long time.
If someone post a message to a forum whose encoding is EUC-JP, Firefox encodes JIS X 0212 characters using SS3 which is not readable for other browsers.
If JIS X 0212 encoded is removed, JIS X 0212 characters will be converted to character references per HTML5 spec.
http://www.w3.org/TR/html5/association-of-controls-and-forms.html#application-x-www-form-urlencoded-encoding-algorithm
I will not remove JIS X 0212 decoder for backward compatibility with previous versions of Firefox and non-HTML5-compliant implementations. This is a similar approach to our Big5 decoder (see bug 310299).
(Assignee)

Updated

8 years ago
status2.0: --- → ?
(Assignee)

Comment 1

8 years ago
Created attachment 479750 [details] [diff] [review]
removal patch
Assignee: smontagu → VYV03354
Status: NEW → ASSIGNED
Attachment #479750 - Flags: review?(smontagu)
Attachment #479750 - Flags: approval2.0?
Attachment #479750 - Flags: review?(smontagu) → review+

Comment 2

8 years ago
Can you outline the risk/reward for this patch, and what kind of testing it ought to receive from nightly and beta testers once it lands?

I do not see any tests for the new encoding behavior, which makes me leery of approving the patch in any case.
(Assignee)

Updated

8 years ago
Depends on: 610267

Comment 3

8 years ago
Comment on attachment 479750 [details] [diff] [review]
removal patch

Clearing approval request until questions are addressed.
Attachment #479750 - Flags: approval2.0?

Comment 4

8 years ago
Background:

The encoding EUC-JP has some variants.
* original Japanese EUC (US-ASCII + JIS X 0201 Katakana + JIS  X 0208)
* EUC-JP defined by IANA (US-ASCII + JIS X 0201 Katakana + JIS  X 0208 + JIS X 0212) 
* CP51932 defined by Microsoft (US-ASCII + JIS X 0201 Katakana + JIS X 0208 + NEC special character + NEC selected IBM extended character)

Current implementation:

Mozilla implements own special encoding (US-ASCII + JIS X 0201 Katakana + JIS X 0208 + NEC special character + NEC selected IBM extended character + JIS X 0212)

Current Problem:

On posting forms, Mozilla sends data whose character set is larger than other implementations.
IANA EUC-JP complient implementations can't process NEC special characters and NEC selected IBM extended characters.
CP51932 comlient implementations can't process JIS X 0212 characters.
So current Mozilla lacks interoperability.

Ideal behavior:

Follow CP51932.
Why follow CP51932 is because HTML5 says pages declared as "EUC-JP" encoding
should be proceeded as CP51932. see Character encoding overrides table in HTML5.
http://www.w3.org/TR/html5/parsing.html#character-encodings-0

But it is inconvenient that Mozilla can't see data posted by previous Mozilla.
So remove JIS X 0212 which is difference between CP51932 and current Mozilla encoding
only from encoder; not remove from decoder.

The patch do this.

See also:

https://bugzilla.mozilla.org/show_bug.cgi?id=310299 is similar problem and did similar solution.
(Assignee)

Updated

8 years ago
Keywords: checkin-needed
http://hg.mozilla.org/projects/cedar/rev/151bbff21acf
Keywords: checkin-needed → html5
Summary: [HTML5] Remove JIS X 0212 support from EUC-JP encoder → Remove JIS X 0212 support from EUC-JP encoder
Whiteboard: fixed-in-cedar

Comment 6

8 years ago
http://hg.mozilla.org/mozilla-central/rev/151bbff21acf
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Whiteboard: fixed-in-cedar
Target Milestone: --- → mozilla2.2
You need to log in before you can comment on or make changes to this bug.