93343 - UTF-16 page displayed incorrectly

Status: VERIFIED FIXED

(Core :: Internationalization, defect)

Description

[Matt Seitz]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.2) Gecko/20010628
BuildID:    2001080203

Instead of rendering the page, Mozilla displays the page source.  The source
appears as ASCII characters with rectangle characters between each ASCII
character.  It appears the page source is in UCS16 format but Mozilla is
treating it as an ASCII text file.

Reproducible: Always
Steps to Reproduce:
1.  Go to www.sellsbrothers.com/tools



Actual Results:  A distorted version of the page source was displayed

Expected Results:  The page source should be interpreted and rendered.

Comment 1

[Boris Zbarsky [:bzbarsky]]

Interesting.... here's what I see with linux build 2001080208.  The page source
is shown.  If I delete the first two bytes of the file, the source is only shown
if charset autodetect is:

Off, Russian, Ukrainian, East Asian, Japanese, 

For other values of autodetect (Korean and the various Chinese ones), the page
is detected as UTF-16LE and shown correctly...

Comment 2

[Roy Yokoyama]

I checked the nsParser code but it's still there.
http://lxr.mozilla.org/seamonkey/source/htmlparser/src/nsParser.cpp#2601

ylong: Is this regression?
I can't reproduce this using NS6.1
However, I see it using trunk 20010730
==== assigning to ftang.========

Comment 3

[Yuying Long]

I guess it's a regression:
I can reproduce it on my Win2k-CN on 07-25 branch build, but 07-19 branch build
is OK.

Comment 4

[Frank Tang]

N4.x display them correctly.

Comment 5

[Boris Zbarsky [:bzbarsky]]

*** Bug 94225 has been marked as a duplicate of this bug. ***

Comment 6

[[e-mail] (away - not reading bugmail)]

reassigning per ftang's request

Comment 7

[[e-mail] (away - not reading bugmail)]

accepting and targeting post-0.9.4. Will clarify with ftang, whether this is 
acceptable...

Comment 8

[Frank Tang]

The meta tag said charset=unicode
It does have a BOM in front. 
move back to ftang to look at

Comment 9

[Frank Tang]

mark it as assign

Comment 10

[Frank Tang]

It looks like vidur's 3.254 check in cause this problem. Vidur, we should only
ignore those UTF-16/32 when we get them from meta tag. We should not ignore them
if we get them from the byte order mark.

 

Comment 11

[Frank Tang]

Attachment: patch

Comment 12

[Frank Tang]

I think this is introduce after n6.1 beta. regression.

harishd- can you r= this
vidur- can you sr= this?

Comment 13

[harishd]

Comment on attachment 53819 [details] [diff] [review]
patch

r=harishd. 

Suggestion: nsICharsetAlias
creation could be delayed.

Comment 14

[vidur (gone)]

Comment on attachment 53819 [details] [diff] [review]
patch

sr=vidur

Comment 15

[Frank Tang]

fixed and check in

Comment 16

[Matt Seitz]

Works for me in build 2001110903

Comment 17

[Yuying Long]

It display fine on 11-16 trunk build, mark it as verified.