180461 - charset auto-detection is disabled because all-chimera.js

Status: RESOLVED FIXED

(Camino Graveyard :: Page Layout, defect)

Description

[sylphied]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.0.1) Gecko/20021114 Chimera/0.6+
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.0.1) Gecko/20021114 Chimera/0.6+

The line in all-chimera.js

pref("intl.charset.detector", "universal_charset_detector");

disable charset auto-detection.
If delete this line, auto-detection will be enabled.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[Greg K.]

So, you're saying that line disables charset detection, and if you remove it,
charset detection is enabled. Sounds fine. What's the problem?

Comment 2

[kaldari]

Apparently, there are some problems with auto-detection (bug 168526 for
example). This is probably why charset auto-detection is no longer the default
setting in Chimera. It'll probably be changed back at some point. Marking INVALID.

Comment 3

[Mike Pinkerton (not reading bugmail)]

wtf? why was this marked invalid?

isn't this still a real issue?

Comment 4

[Brian Ryner (not reading)]

I don't know enough about this to say whether this pref should be enabled.

Comment 5

[S Woodside]

Marking depends on bug 168526. NB the disk cache also stores the detected/set
encoding! :-)

Someone describe the scope of this bug and the impact on browsing please?

Comment 6

[Ludovic Hirlimann [:Usul]]

Reporter would you answer's simon's question ?
Is this till true with newer builds ?
Is it camlino specific or does Mozilla behaves the same way ?

Comment 7

[Jasper]

I did some little investigation into the rather vage report. What happens is this:

1) When going to http://www.geocities.co.jp/, which is a website that doesn't
have a DOCTYPE tag, the site is displayed as a Wetsern ISO webpage (while it
should be a Japanese EUC page). Which in it's turn causes the webpage to look
like un readable crap.
2) When removing the given line from the all-chimera.js file Camino displays the
given webpage correct as a Japanese EUC page.

Camino should automaticalluy detect what charset should be used if a page
doesn't have a DOCTYOE tag.

When removing the all-chimera.js line I haven't encountered any problems with
other sites so far. It appears that removing the line activates the charset
detector. Which seems wrong.

Mozilla 1.5a behaves the same as Camino. In Mozilla charset auto detection is
also turned off by default. When setting it to "universal" instead of "off", in
charecter coding menu, the the page displays correctly.

Perhaps there should be an option added to the Text Encoding menu in Camino
allowing the user to trun charset auto detection on and off just as Mozilla
does. (why is the charset auto detect turned off by default?)

Comment 8

[Jasper]

Correction: I tested this in Mozilla 1.5b. 

Sorry for the spam.

Comment 9

[S Woodside]

perhaps this bug should not depends on bug 168526, since that's a ?minor?
problem, jasper claims not to have noticed any sites with this problem
(presumably you didn't try the site on that bug though?). I'll try turning the
auto-detect on and see if I notice any problems.

Comment 10

[S Woodside]

Re comment #1: One problem is that the line in
Camino.app/MacOS/defaults/pref/all-chimera.js is labelled "// turn on universal
character detection" but in fact, it turns OFF detection.

Comment 11

[Brian Ryner (not reading)]

-> pinkerton to investigate; I'm without a Mac for the time being.

Comment 12

[Jasper]

Re: Comment #9
When visiting the URL given in Bug 168526 and removing the all-chimera.js line
the webpage indeed looks as described in Bug 168526. In which case it seems that
charset auto detection does not work on that particular page.

Re: Comment #10
That is exactly what is going on.

(The problem is not that big since we are able to change the charset in Camino
since a couple of months. It would be nice to be able to perhaps give the user a
more eccesible way to enable/disable charset auto-detection from within Camino.)

Comment 13

[Mike Pinkerton (not reading bugmail)]

how very bizarre. who's left that understands what this pref should do?

Comment 14

[Josh Aas]

As for comment #10, the reporter says that the line turns off "auto-detection"
which may be entirely different than "universal_charset_detector". Based on
testing, I think it probably is different. "universal_charset_detector" is
probably superceding the auto-detection when its enabled.

When I don't comment out that line, http://www.geocities.co.jp/ displays
incorrectly. When I do comment out the line, it displays incorrectly. I don't
get any problems anywhere else with the line commented out.

- Firebird from 1/4 displays that link incorrectly.
- Mozilla build 20040104 displays that link incorrectly.

Whatever that universal_charset_detector is its not doing the job on some
Japanese pages. Maybe the best move is to get rid of that line for a while
before 0.8 and see what we run into with a bunch of testers.

Comment 15

[Mike Pinkerton (not reading bugmail)]

f