320024 - conflict between MIME charset and meta charset should be better resolved

Status: RESOLVED WONTFIX

(Core :: Internationalization, enhancement)

Description

[Eyal Rozenberg]

A possible solution to the issue raised in bug 255738 and bug 214952: Instead of always preferring meta over MIME or the other way around, apply the following short-circuit logic:

1. If the MIME charset isn't one of: {iso-8859-1, US-ASCII}, use the MIME charset.
2. Guess the charset using the auto-detection algorithm.
3. If the auto-detection agrees with the meta charset, use the meta charset.
4. Use the MIME charset.

For a motivation of the two values in item 1, see bug 260728.

Comment 1

[Jesse Ruderman]

I'd expect this to be wontfix for the same reason as bug 214952...

Comment 2

[Eyal Rozenberg]

(In reply to comment #1)

Not at all. This is a middle-ground suggestion; see
https://bugzilla.mozilla.org/show_bug.cgi?id=214952#c15
https://bugzilla.mozilla.org/show_bug.cgi?id=214952#c18

Comment 3

[Jungshik Shin]

sorry to pick nits:
re: comment #0
You should have used '(charset specified in) HTTP header' (respect HTTP header) and '(charset specfied with) meta tag'  rathaer than 'MIME' and 'meta'. Whether it's specifed in HTTP header or meta tag, it's always MIME charset.

BTW, you can just use 'bug xyz comment i' in place of a full url. 

Comment 4

[Jesse Ruderman]

It violates the spec in fewer situations than your previous suggestions, but that doesn't make it "middle-ground".  I believe it still goes directly against what the specs say.

Comment 5

[Eyal Rozenberg]

I already explained why it does not violate the spec.

Comment 6

[Eyal Rozenberg]

Plus, debate it on the merits, then we'll talk about spec compliance.

Comment 7

[Henri Sivonen (:hsivonen)]

We implement the spec now.

Comment 8

[Eyal Rozenberg]

(In reply to Henri Sivonen (:hsivonen) from comment #7)
Do you implement it as I argue you should, here:
https://bugzilla.mozilla.org/show_bug.cgi?id=214952#c8
?

If so, then WORKSFORME is valid, if not, why not?

Comment 9

[Henri Sivonen (:hsivonen)]

Multiple browsers implement the spec, and it is believed that deviating from the spec would break stuff. The situation isn't ideal in the light of Ruby's Postulate (http://www.intertwingly.net/slides/2004/devcon/69.html), but rocking the boat at this point isn't worthwhile.

Comment 10

[Eyal Rozenberg]

(In reply to Henri Sivonen (:hsivonen) from comment #9)

This does not actually address my question. I believe my suggestion is compatible with the spec (although frankly I did not look for changes from HTML 4 to HTML 5 yet) - as per the discussion in the comments for bug 214952. Can you explain how/why meeting the spec conflicts with the suggestion here?

I realize I'm being somewhat pedantic, but this is somewhat of a fine point, and you have closed my bug, so - please indulge me.

Comment 11

[Henri Sivonen (:hsivonen)]

"The spec" means https://html.spec.whatwg.org/ . HTML4 is not relevant.

https://html.spec.whatwg.org/#determining-the-character-encoding and https://encoding.spec.whatwg.org/#decode taken together specify the order. (The BOM takes the highest precedence, then HTTP, then <meta>.)