168737 - [FIX]Syntax coloring not working for UTF-16 encoded pages.

Status: RESOLVED FIXED

(Core Graveyard :: View Source, defect, P1)

Description

[Niklas Dougherty]

Load http://www.student.lu.se/~kin02ndo/ (and any of the links there), and then
view the source. The page syntax is all black rather than colored.

Might be the byte order mark messing up something.

(Neither does wrapping of long lines in the source work, but that seems to be a
separate bug pertaining to all encodings.)

Comment 1

[Greg K.]

Confirmed using FizzillaCFM/2002091014 on 10.1.5.

Reassigning to View Source and adding intl keyword.

(Niklas, FYI, you have a couple of HTML syntax errors at that URL, including a
misquoted anchor attribute value (missing trailing quote) and an extraneous </div>.)

Comment 2

[Niklas Dougherty]

Thanks. Damn W3C validator doesn't recognize UTF-16. Could that be a tech
evangelism bug?

Comment 3

[Boris Zbarsky [:bzbarsky]]

Attachment: Stupid fix.

Comment 4

[Boris Zbarsky [:bzbarsky]]

caillon, could you review?

Comment 5

[Christopher Aillon (sabbatical, not receiving bugmail)]

Comment on attachment 102425 [details] [diff] [review]
Stupid fix.

r=caillon

Comment 6

[Doron Rosenberg (IBM)]

shouldn't that be added after the licence block?

Comment 7

[Boris Zbarsky [:bzbarsky]]

No, because that would be illegal CSS.  The @charset rule must be the first
thing in the file; no comments, whitespace, whatever allowed before it.

If you think about it for 10 seconds, you should realize why that is (hint: how
do I parse the comment as a comment without knowing the encoding?)

Comment 8

[Christian :Biesinger (don't email me, ping me on IRC)]

>how do I parse the comment as a comment without knowing the encoding?

well... how do you parse the @charset without knowing the encoding? or should
that always be in US-ASCII?

Comment 9

[Boris Zbarsky [:bzbarsky]]

You parse it by knowing that it's 7-bit-clean data possibly encoded in a 2 or 4
byte encoding.  Why ask me?  Just look it up --
http://lxr.mozilla.org/seamonkey/source/content/html/style/src/nsCSSLoader.cpp#611

The key is that it starts with a known useful string -- "@charset"

Comment 10

[rbs]

Comment on attachment 102425 [details] [diff] [review]
Stupid fix.

sr=rbs

Comment 11

[Asa Dotzler [:asa]]

Comment on attachment 102425 [details] [diff] [review]
Stupid fix.

a=asa for checkin to 1.2beta (on behalf of drivers)

Comment 12

[Christopher Aillon (sabbatical, not receiving bugmail)]

Just FYI, for those who care:

http://www.w3.org/TR/CSS2/syndata.html#q23
http://www.w3.org/TR/CSS21/syndata.html#q19

Fourth paragraph explains the @charset at-rule.

Comment 13

[Boris Zbarsky [:bzbarsky]]

checked in