746911 - (encoding) [meta] Implement the Encoding Standard

Status: NEW

(Core :: Internationalization, enhancement)

Description

[Henri Sivonen (:hsivonen)]

Gecko should implement http://dvcs.w3.org/hg/encoding/raw-file/tip/Overview.html for better interoperability and for reduced brokenness.

This involves:
 * Changing the charset alias list to match the spec
 * Changing the Unicode converter API to be able to signal the end of the stream
 * Fixing various converters to signal errors per spec
 * Fixing various converters to behave per spec
 * Removing some converters

Comment 1

[Axel Hecht [:Pike]]

Is there any documentation on why the choices made in the encoding standard are as they are? In particular on the mix between windows and iso latin encodings?

Comment 2

[Henri Sivonen (:hsivonen)]

(In reply to Axel Hecht [:Pike] from comment #1)
> Is there any documentation on why the choices made in the encoding standard
> are as they are?

The set of encodings is based on researching the commonality between browsers. The basic assumption is that Web content wants to work in multiple browsers, so if only one browser supports a given fringe encoding, the encoding is probably not in significant use. IIRC, actual documentation of test results is somewhere in www-archive.

> In particular on the mix between windows and iso latin
> encodings?

When a windows encoding is a superset of an ISO encoding, the encoding standard retains the windows encoding and makes the ISO labels aliases thereof. In practice, the ISO labels already invoke decoders that are actually decoders for the corresponding windows encoding and IE reports the windows label (and has notable marketshare in many places).

In the cases where there's no windows superset for an ISO encoding, the Encoding Standard uses the ISO-8859-x label as the preferred name.