457845 - Firefox 3.0.3: responseText of XMLHttpRequest receives wrong characters from server

Status: RESOLVED INCOMPLETE

(Core :: DOM: Core & HTML, defect)

Description

[Monica Bettoli]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; it; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; it; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3

Until yesterday I used Firefox 2.0.0.17 and all applications worked good.
My web applications use XMLHttpRequest. I receive from server the answer in responseText and the string in responseText is: rtcd=0¶
Today I installed Firefox 3.0.3 and the same applications receive in responseText this wrong string: rtcd=0¶ 
Firefox 3.0.3 add the character Â
The same happens if server answer the characters £ and §, Firefox add the character  before the correct character.
The most of our web applications cannot work with Firefox 3.0.3, my company cannot use this new version.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[Monica Bettoli]

Attachment: example of my application

This attachment contains the source html of my application

Comment 2

[Monica Bettoli]

I have the same problem using Firefox 3.0.3 for linux

Comment 3

[Monica Bettoli]

Attachment: example of my application

This attachment contains the source html of my application and the print-screen of the responseText using Firefox 2.0.17 and 3.0.3

Comment 4

[Boris Zbarsky [:bzbarsky]]

Monica, could you please tell me what encoding your server sends the string in?  What headers does it send?

Comment 5

[Monica Bettoli]

Hi Boris, my server use default charset ISO-8859-1.
My document.characterSet is ISO-8859-1.
Yesterday I have tested different situations using firefox 2 and firefox 3.
I add you the attachment with the results I have obtained.
Using test case 4 I obtain the correct characters in both versions, but my applications lost compatibility with IE that doesn't recognize overrideMimeType and no longer works.
Why using the same application with the same setRequestHeader and overrideMimeType , the results using version 2 and 3 are different?
How can I have correct results from firefox 3 without lost compatibility with IE?

Comment 6

[Monica Bettoli]

Attachment: Tests case

This document contains different test case and the result I have obtained.

Comment 7

[Boris Zbarsky [:bzbarsky]]

> Hi Boris, my server use default charset ISO-8859-1.

That doesn't answer my question.  It sure looks to me like it's sending back UTF-8 and claiming it's ISO-8859-1 or something.

Does it base the bytes it sends on the bytes it receives?  Note that Firefox is sending it UTF-8.  Does it send the same bytes to Firefox 2 and Firefox 3?

Could you perhaps create an HTTP log of the failing case following the instructions at http://www.mozilla.org/projects/netlib/http/http-debugging.html and attach it to this bug?  And do the same for the same testcase in Firefox 2?

Comment 8

[Monica Bettoli]

Attachment: Log about actual set of my applications using FF2

The log contains these steps using Firefox 2.0.17:
1) open firefox 2.0.17
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/vq10rh1h.cgi

look to "example of my application" attachment to see the setting.

Comment 9

[Monica Bettoli]

Attachment: Log about actual set of my applications using FF3

The log contains these steps using Firefox 3.0.3:
1) open firefox 3.0.3
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "example of my application" attachment to see the setting.

Comment 10

[Monica Bettoli]

Attachment: Log about test case 1 using FF2

The log contains these steps using Firefox 2.0.17:
1) open firefox 2.0.17
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "test case" attachment (case 1) to see the setting.

Comment 11

[Monica Bettoli]

Attachment: Log about test case 1 using FF3

The log contains these steps using Firefox 3.0.3:
1) open firefox 3.0.3
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "test case" attachment (case 1) to see the setting.

Comment 12

[Monica Bettoli]

Attachment: Log about test case 2 using FF2

The log contains these steps using Firefox 2.0.17:
1) open firefox 2.0.17
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "test case" attachment (case 2) to see the setting.

Comment 13

[Monica Bettoli]

Attachment: Log about test case 2 using FF3

The log contains these steps using Firefox 3.0.3:
1) open firefox 3.0.3
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "test case" attachment (case 2) to see the setting.

Comment 14

[Monica Bettoli]

Attachment: Log about test case 3 using FF2

The log contains these steps using Firefox 2.0.17:
1) open firefox 2.0.17
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "test case" attachment (case 3) to see the setting.

Comment 15

[Monica Bettoli]

Attachment: Log about test case 3 using FF3

The log contains these steps using Firefox 3.0.3:
1) open firefox 3.0.3
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "test case" attachment (case 3) to see the setting.

Comment 16

[Monica Bettoli]

Attachment: Log about test case 4 using FF2

The log contains these steps using Firefox 2.0.17:
1) open firefox 2.0.17
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "test case" attachment (case 4) to see the setting.

Comment 17

[Monica Bettoli]

Attachment: Log about test case 4 using FF3

The log contains these steps using Firefox 3.0.3:
1) open firefox 3.0.3
2) open bookmark and choose /cgigrp/testwnd.cgi
3) click submit and execute XMLHttpRequest for /cgigrp/testwndh.cgi

look to "test case" attachment (case 4) to see the setting.

Comment 18

[Monica Bettoli]

Mistake in description of attachment id=341592 the XMLHttpRequest is for /cgigrp/testwndh.cgi (not for /cgigrp/vq10rh1h.cgi). Sorry.

Comment 19

[Boris Zbarsky [:bzbarsky]]

So Fx2 sends:

  Content-Type: application/x-www-form-urlencoded; charset=ISO-8859-1

Fx3 sends:

  Content-Type: application/x-www-form-urlencoded; charset=UTF-8

Your server responds to Fx2 with:

  Content-Type: text/html; charset=ISO-8859-1

and responds to Fx3 with:

  Content-Type: text/html; charset=ISO-8859-1;text/html; charset=ISO-8859-1;erxt/html; charset=ISO-8859-1;erst/html

I would guess the issue is the server messing up the decoding of the urlencoded data somehow (e.g. assuming that the charset param says something about the decoded data, not the encoded data).

Comment 20

[Boris Zbarsky [:bzbarsky]]

Those sends/receives pairs were for the "actual set of my applications" attachments.

Comment 21

[Monica Bettoli]

In test case 1:
setRequestHeader("Content-Type", "application/x-www-form-urlencoded;
charset=ISO-8859-1");

FF2 sends:
   Content-Type: application/x-www-form-urlencoded; charset=ISO-8859-1

FF3 sends:
   Content-Type: application/x-www-form-urlencoded; charset=UTF-8

Why FF3 sends UTF-8 when setRequestHeader is ISO-8859-1?

Comment 22

[Boris Zbarsky [:bzbarsky]]

Because the data it's actually sending is in UTF-8, just like in Fx2.  Of course in this case the data is all-ASCII, so there is no difference in terms of what bytes are sent.  But it sounds like the server is misusing the charset param to mean the encoding of the bytes once the data is unescaped, as I said in comment 19.  That's not what the charset param means.

Comment 23

[Alexander]

var req = new XMLHttpRequest();  
		var mode = false
		req.open('GET', url, mode);  
		req.overrideMimeType('text/plain; charset=x-user-defined');  
		req.onreadystatechange = function() {
			if(req.readyState == 3) {
				alert(req.responseText.charCodeAt(0) & 0xff);
				
			}
		}
		req.send(null);
		if(!mode){
			alert('NSYNC');
			alert(req.responseText.charCodeAt(0) & 0xff);
		}	

Url may points to any binary file with first byte > ANSI range
If var mode = false this function dispays the same correct result in Firefox 2.0.0.20 and Firefox 3.0.5.
If var mode = true Firefox 2 works in the same correct way when Firefox 3 dispayes incorrect byte code.

Comment 24

[Boris Zbarsky [:bzbarsky]]

That doesn't sounds related to this bug.  Please file a new bug on that, and cc me, ok?

Comment 25

[Alluri Hemanth]

Comment on attachment 341100 [details]
example of my application

is it sill coming..? i'm unable to download the attachment..

Comment 26

[Anne (:annevk)]

There's not enough information here to reproduce the problem.