138862 - NY Times reported as Simplified Chinese with Auto-detect on

Status: VERIFIED WORKSFORME

(Core :: Internationalization, defect)

Description

[Greg Lloyd]

With auto detect (Chinese) enabled, encoding of NYT is reported as Simplified
Chinese (gb18030). This mangles special characters such as bullets, and shows
ugly scaled text for p13 serif (times). 

Seems to be a regression with 1.0RC1

Comment 1

[Greg Lloyd]

With auto-detect set to Universal, I see NYT encoding is reported as Central
European (ISO 8859-2).

I installed 1.0RC1 today by replacing the Mac OS X package (but retaining the
preference resources installed by earlier Mozilla Mac OS X builds).

Comment 2

[Yuying Long]

Confirm when auto-detect Chinese, it detect as gb18030(it's not a good
auto-detect option though) and auto-detect All detect as ISO 8859-2.

Reassign to shanjian.

Comment 3

[Roy Yokoyama]

assign to shanjian

Comment 4

[Shanjian Li]

The home page has been updated, so I can no longer reproduce the problem. But
anyway, I could understand what's happening. GB18030 covers a lot more code
points, and that make it possible to interpret the page as gb18030. There is no
much we can do in this situation. Autodetection can never be prefect. 

The universal detector problem have been fixed in trunk. Branch landing is pending.

Comment 5

[Yuying Long]

With auto-detect Universal, the NY times is detected as latin1 iso-8859-1.  And
for auto-detect Chinese we might not able work correct on those kind of case
right now per Shanjian's comment.

Mark as verified.