220476 - Charset autodetection detects big5 page to be gb18030

Status: RESOLVED WORKSFORME

(Core :: Internationalization, defect)

Description

[Bill Mason]

Spun off from bug 220438 per bz's request.  Relevant comment from that bug:

When I enter a page without charset infomation in either HTTP headers and
HTML meta section, the default character encoding in my preference setting
should apply.  But it's not.  It seems that Mozilla 1.4+ interprets the
character encoding by itself with some encoding-guessing subroutine.  But that
subroutine makes mistakes all the times.  It misguessed Traditional Chinese
(Big5) pages as Simplified Chinese (GB13080) or Korean all the time.  I have to
switch the encoding back to Traditional Chinese (Big5) for every page on some
websites, and there is no shortcut key to do that.  More worse, some websites
put a lot of Big5 plain text files on them.  Plain text files has no encoding
infomation at all, so Mozilla simply put them as Simplified Chinese every time.

Comment 1

[Jungshik Shin]

There's no SC character encoding called gb13080. It's GB18030.Moreover, the bug
report shouldn't have been just copied from bug 220438. This bug is NOT about
Mozilla ignoring the default character encoding BUT about charset-autodetector's
failure to detect Big5. 

BTW, which auto-detector did you use? With Chinese (instead of 'universal'), the
page in the URL field got autodetected as Big5. 

Comment 2

[Mike Cowperthwaite]

Bill Mason, you did not response to comment 1.

Bug 181344 is specifically about the Universal detector inadvertantly detecting 
Western pages as GB18030.

Bug 178495 is about the Universal detector detecting a "SimpChinese" page as 
Korean.

Pick one of these and dupe this bug to it, or explain why it's not a dupe.

Comment 3

[Bill Mason]

I wrote this bug to answer bz's request in bug 220438 comment 5.

I have never seen this issue, tested this issue, and am really not wrapped up in
what happens to it.

It might help if you read the cited bug first before asking me for anything more.

Comment 4

[Mike Cowperthwaite]

Sorry to be a pest, Bill.

The URL listed above is detected as Big5 with both the Universal and Chinese 
autodectors, in
  Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8a3) Gecko/20040714

I'm inclined to WFM the bug, but instead I'll CC: the reporter of the original 
bug (imacat) to get a comment.

Comment 5

[imacat]

Hi.  This imacat, the original submitter for bug #220438 last year.  I'm wonder
why it is mentioned after a year.  However, after making some tests I think this
bug should be solved on my Mozila 1.6, win32, Traditional Chinese build.  Thank
you for your effort and attention on this.

Comment 6

[Mike Cowperthwaite]

(In reply to comment #5)
> after making some tests I think this
> bug should be solved on my Mozila 1.6, win32, Traditional Chinese build.

I interpret that to mean "this bug *has been* solved ... in 1.6"

=>WFM