246651 - XMLHttpRequest doesn't calculate Content-Length correctly

Status: RESOLVED FIXED

(Core :: XML, defect)

Description

[william]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7rc1) Gecko/20040528
Build Identifier: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7rc1) Gecko/20040528

Javascript XMLRPC clients based on Mozilla cannot communicate with some XMLRPC
servers. The problem seems to come from the way Mozilla calculates the
Content-Length related to XML messages it sends. It simply doesn't count the
last "\n\r" at the end of the messages. As a result, the Content-Length is 2
units too short.
I included the content of two traces. The first was done using Mozilla and the
second was obtained using perl LWP POST script (POST http://localhost:8080/). 

Reproducible: Always
Steps to Reproduce:
1. Start an XMLRPC server on the port 8080 (or start netcat like this: "nc -l -p
8080")
2. Put the following html/javascript snippet in a file 
3. Point your Mozilla browser to it
4. Capture the traffic (with ethereal or tcpdump) 

Actual Results:  
The Content-Length has a value of 81.

Expected Results:  
The Content-Length should have a value of 83.

* HTML/JAVASCRIPT SNIPPET
<script type="application/x-javascript">
netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect
UniversalBrowserAccess');
var xmlhttp = Components.classes['@mozilla.org/xmlextras/xmlhttprequest;1']
.createInstance(Components.interfaces['nsIXMLHttpRequest']);
xmlhttp.open('POST', 'http://localhost:8080/');
xmlhttp.send("<?xml
version=\"1.0\"?><methodCall><methodName>test.query</methodName></methodCall>");
</script>

* NETWORK TRACES
(Using Mozilla 1.7rc1)
POST / HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7rc1) Gecko/20040528
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Content-Type: text/xml
Content-Length: 81
Pragma: no-cache
Cache-Control: no-cache

<?xml version="1.0"?><methodCall><methodName>test.query</methodName></methodCall>

00000000  50 4f 53 54 20 2f 20 48  54 54 50 2f 31 2e 31 0d POST / H TTP/1.1.
00000010  0a 48 6f 73 74 3a 20 6c  6f 63 61 6c 68 6f 73 74 .Host: l ocalhost
00000020  3a 38 30 38 30 0d 0a 55  73 65 72 2d 41 67 65 6e :8080..U ser-Agen
00000030  74 3a 20 4d 6f 7a 69 6c  6c 61 2f 35 2e 30 20 28 t: Mozil la/5.0 (
00000040  58 31 31 3b 20 55 3b 20  4c 69 6e 75 78 20 78 38 X11; U;  Linux x8
00000050  36 5f 36 34 3b 20 65 6e  2d 55 53 3b 20 72 76 3a 6_64; en -US; rv:
00000060  31 2e 37 72 63 31 29 20  47 65 63 6b 6f 2f 32 30 1.7rc1)  Gecko/20
00000070  30 34 30 35 32 38 0d 0a  41 63 63 65 70 74 3a 20 040528.. Accept: 
00000080  74 65 78 74 2f 78 6d 6c  2c 61 70 70 6c 69 63 61 text/xml ,applica
00000090  74 69 6f 6e 2f 78 6d 6c  2c 61 70 70 6c 69 63 61 tion/xml ,applica
000000A0  74 69 6f 6e 2f 78 68 74  6d 6c 2b 78 6d 6c 2c 74 tion/xht ml+xml,t
000000B0  65 78 74 2f 68 74 6d 6c  3b 71 3d 30 2e 39 2c 74 ext/html ;q=0.9,t
000000C0  65 78 74 2f 70 6c 61 69  6e 3b 71 3d 30 2e 38 2c ext/plai n;q=0.8,
000000D0  69 6d 61 67 65 2f 70 6e  67 2c 2a 2f 2a 3b 71 3d image/pn g,*/*;q=
000000E0  30 2e 35 0d 0a 41 63 63  65 70 74 2d 4c 61 6e 67 0.5..Acc ept-Lang
000000F0  75 61 67 65 3a 20 65 6e  2d 75 73 2c 65 6e 3b 71 uage: en -us,en;q
00000100  3d 30 2e 35 0d 0a 41 63  63 65 70 74 2d 45 6e 63 =0.5..Ac cept-Enc
00000110  6f 64 69 6e 67 3a 20 67  7a 69 70 2c 64 65 66 6c oding: g zip,defl
00000120  61 74 65 0d 0a 41 63 63  65 70 74 2d 43 68 61 72 ate..Acc ept-Char
00000130  73 65 74 3a 20 49 53 4f  2d 38 38 35 39 2d 31 2c set: ISO -8859-1,
00000140  75 74 66 2d 38 3b 71 3d  30 2e 37 2c 2a 3b 71 3d utf-8;q= 0.7,*;q=
00000150  30 2e 37 0d 0a 4b 65 65  70 2d 41 6c 69 76 65 3a 0.7..Kee p-Alive:
00000160  20 33 30 30 0d 0a 43 6f  6e 6e 65 63 74 69 6f 6e  300..Co nnection
00000170  3a 20 6b 65 65 70 2d 61  6c 69 76 65 0d 0a 43 6f : keep-a live..Co
00000180  6e 74 65 6e 74 2d 54 79  70 65 3a 20 74 65 78 74 ntent-Ty pe: text
00000190  2f 78 6d 6c 0d 0a 43 6f  6e 74 65 6e 74 2d 4c 65 /xml..Co ntent-Le
000001A0  6e 67 74 68 3a 20 38 31  0d 0a 50 72 61 67 6d 61 ngth: 81 ..Pragma
000001B0  3a 20 6e 6f 2d 63 61 63  68 65 0d 0a 43 61 63 68 : no-cac he..Cach
000001C0  65 2d 43 6f 6e 74 72 6f  6c 3a 20 6e 6f 2d 63 61 e-Contro l: no-ca
000001D0  63 68 65 0d 0a                                   che..
000001D5  0d 0a 3c 3f 78 6d 6c 20  76 65 72 73 69 6f 6e 3d ..<?xml  version=
000001E5  22 31 2e 30 22 3f 3e 3c  6d 65 74 68 6f 64 43 61 "1.0"?>< methodCa
000001F5  6c 6c 3e 3c 6d 65 74 68  6f 64 4e 61 6d 65 3e 74 ll><meth odName>t
00000205  65 73 74 2e 71 75 65 72  79 3c 2f 6d 65 74 68 6f est.quer y</metho
00000215  64 4e 61 6d 65 3e 3c 2f  6d 65 74 68 6f 64 43 61 dName></ methodCa
00000225  6c 6c 3e 0d 0a                                   ll>..

(Using perl LWP POST script)
POST / HTTP/1.1
Connection: close
Host: localhost:8080
User-Agent: lwp-request/2.01
Content-Length: 82
Content-Type: application/x-www-form-urlencoded

<?xml version="1.0"?><methodCall><methodName>test.query</methodName></methodCall>

00000000  50 4f 53 54 20 2f 20 48  54 54 50 2f 31 2e 31 0d POST / H TTP/1.1.
00000010  0a 43 6f 6e 6e 65 63 74  69 6f 6e 3a 20 63 6c 6f .Connect ion: clo
00000020  73 65 0d 0a 48 6f 73 74  3a 20 6c 6f 63 61 6c 68 se..Host : localh
00000030  6f 73 74 3a 38 30 38 30  0d 0a 55 73 65 72 2d 41 ost:8080 ..User-A
00000040  67 65 6e 74 3a 20 6c 77  70 2d 72 65 71 75 65 73 gent: lw p-reques
00000050  74 2f 32 2e 30 31 0d 0a  43 6f 6e 74 65 6e 74 2d t/2.01.. Content-
00000060  4c 65 6e 67 74 68 3a 20  38 32 0d 0a 43 6f 6e 74 Length:  82..Cont
00000070  65 6e 74 2d 54 79 70 65  3a 20 61 70 70 6c 69 63 ent-Type : applic
00000080  61 74 69 6f 6e 2f 78 2d  77 77 77 2d 66 6f 72 6d ation/x- www-form
00000090  2d 75 72 6c 65 6e 63 6f  64 65 64 0d 0a 0d 0a    -urlenco ded....
0000009F  3c 3f 78 6d 6c 20 76 65  72 73 69 6f 6e 3d 22 31 <?xml ve rsion="1
000000AF  2e 30 22 3f 3e 3c 6d 65  74 68 6f 64 43 61 6c 6c .0"?><me thodCall
000000BF  3e 3c 6d 65 74 68 6f 64  4e 61 6d 65 3e 74 65 73 ><method Name>tes
000000CF  74 2e 71 75 65 72 79 3c  2f 6d 65 74 68 6f 64 4e t.query< /methodN
000000DF  61 6d 65 3e 3c 2f 6d 65  74 68 6f 64 43 61 6c 6c ame></me thodCall
000000EF  3e 0a                                            >.

* Tested configurations
x86_64: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7rc1) Gecko/20040528 
x86:    Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040308
x86:    Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113
x86:    Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040608

Comment 1

[Boris Zbarsky [:bzbarsky]]

Yep. See
http://lxr.mozilla.org/seamonkey/source/extensions/xmlextras/base/src/nsXMLHttpRequest.cpp#1047
 (that should just do "charlength + 2", I think).

Comment 2

[Boris Zbarsky [:bzbarsky]]

Attachment: Patch

Comment 3

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 158466 [details] [diff] [review]
Patch

This patch just changes us to let HTTP handle the content-length and
content-type headers the way it likes (and insert its own leading CRLF as
needed).

I don't really know offhand how to test this; any suggestions would be welcome.

Comment 4

[Johnny Stenback (:jst)]

Comment on attachment 158466 [details] [diff] [review]
Patch

Looks good to me, sr=jst

Comment 5

[Boris Zbarsky [:bzbarsky]]

Fixed on trunk.  Please test this with tomorrow's builds to verify?

Comment 6

[Boris Zbarsky [:bzbarsky]]

Attachment: One more place where I forgot to remove +4

Comment 7

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 159880 [details] [diff] [review]
One more place where I forgot to remove +4

Darin, would you review?  This is just an oversight from the earlier patch in
this bug...  without this we send 4 random bytes (well, a null and 3 random
bytes) in our postdata...

Asking for 1.8a4 approval -- this is a very safe change that's needed to keep
us from sending garbage at the tail of our XMLHttpRequests.

Comment 8

[Asa Dotzler [:asa]]

Comment on attachment 159880 [details] [diff] [review]
One more place where I forgot to remove +4

a=asa for 1.8a4 checkin.

Comment 9

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 159880 [details] [diff] [review]
One more place where I forgot to remove +4

Checked in on trunk for 1.8a4.

Comment 10

[Boris Zbarsky [:bzbarsky]]

*** Bug 258105 has been marked as a duplicate of this bug. ***

Comment 11

[Christian :Biesinger (don't email me, ping me on IRC)]

*** Bug 283428 has been marked as a duplicate of this bug. ***

Comment 12

[Christian :Biesinger (don't email me, ping me on IRC)]

bug 283428 reported this on the 1.7 branch, should this be fixed there as well?

Comment 13

[Boris Zbarsky [:bzbarsky]]

It's not really critical enough to bother, imo...  If people want it, I can do a
branch patch, of course.

Comment 14

[Doron Rosenberg (IBM)]

I doubt a branch patch is needed.

Comment 15

[Bob Clary [:bc] (inactive)]

*** Bug 285734 has been marked as a duplicate of this bug. ***

Comment 16

[Anand Aiyer]

*** Bug 297185 has been marked as a duplicate of this bug. ***