182796 - sony-europe.com - FF FE on UTF-8 page, Mozilla displays code

Status: RESOLVED FIXED

(Tech Evangelism Graveyard :: Other, defect)

Description

[Devoti Paolo]

User-Agent:       Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.3a) Gecko/20021128
Build Identifier: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.3a) Gecko/20021128

the page is not the usual displayed html source caused by wrong mime type there
is also an encoding issue. Page works with IE
If is an evangelize matter http://www.sony-europe.com/ is worth some effort

Reproducible: Always

Steps to Reproduce:
1.open http://www.sony-europe.com/cds/showpage.asp?nodeid=90734

Actual Results:  
strange stuff is displayed

Expected Results:  
page displayed

Comment 1

[Frankie]

The page starts with hex chars "FF FE", which indicates little-endian byte order
UTF-16. However, the page is sent as UTF-8.

Although this is technically Sony's fault, I believe that Mozilla should handle
this situation more gracefully by allowing BOM Quirks Mode. Two mistaken bytes
should not prevent the entire page from displaying.

% curl -I http://www.sony-europe.com/cds/showpage.asp\?nodeid=90734
HTTP/1.1 200 OK
Server: Microsoft-IIS/4.0
Date: Sun, 23 Feb 2003 02:09:29 GMT
Content-Length: 4592
Content-Type: text/html; Charset=UTF-8
Set-Cookie: ASPSESSIONIDGGQGGQCB=CBHLJHGCENAOLCNHFEBFALIC; path=/
Cache-control: private

Comment 2

[Simon Montagu :smontagu]

If bug 183354 gets fixed, the charset detector should be able to handle cases
like this.

Comment 3

[Alexey Chernyak]

I don't think bug 183354 will fix this.
This is a case of wrong charset being specified in HTTP header.
The document is UTF-16LE, but the HTTP header insists it's UTF-8.
So to display the page, Mozilla would have to override the charset specified in
header, and as far as I know that would be an incorrect behaviour (unless done
by user's request).
At the moment Mozilla does what's right.

bug 183354 fixes the Universal Charset Autodetector. As far as I understand,
this code kicks in only when charset is not specified in HTTP header.

However bug 68738 might have some effect on this, for it checks the BOM on
Parser level. However I'm not sure in what cases that code gets executed.

This is an interesting testpage, and it would be interesting to see how
Mozilla's behaviour will be affected by these 2 patches. But this belongs to
Evangelism.

Comment 4

[Jungshik Shin]

I agree with Alexey. This is an edge(and evngelism) case and 
Sony should fix this non-sense right away.
I'm not sure how Mozilla should handle this case without
violating the standard. Adding a user-preference (not available
in UI) to give a higher precedence to autodetectors(including
BOM detection) than to http header?? 
 

Comment 5

[Simon Montagu :smontagu]

Another problem is that right now the UTF-16 encodings are not readily available
from any menu, so a user will have a problem overriding manually.

Comment 6

[Simon Montagu :smontagu]

To evangelism anyway.

Comment 7

[Simon Montagu :smontagu]

.

Comment 8

[Bob Clary [:bc] (inactive)]

move tristans bugs to other

Comment 9

[Alexey Chernyak]

page no longer exists.
dependencies been fixed.