Last Comment Bug 475444 - HTTP standard violation, Content-length header appears to be ignored under some circumstances
: HTTP standard violation, Content-length header appears to be ignored under so...
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: Networking: HTTP (show other bugs)
: unspecified
: x86 Windows XP
: -- normal (vote)
: mozilla17
Assigned To: Michal Novotny (:michal)
:
Mentors:
http://www.fdn.fr/~arenevier/bugs/moz...
Depends on: 484027
Blocks:
  Show dependency treegraph
 
Reported: 2009-01-26 16:18 PST by zextra
Modified: 2012-08-16 17:58 PDT (History)
10 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
patch v1 (1.23 KB, patch)
2009-03-17 18:06 PDT, Michal Novotny (:michal)
bzbarsky: review+
cbiesinger: superreview+
Details | Diff | Review
patch v2 (2.25 KB, patch)
2009-04-06 17:58 PDT, Michal Novotny (:michal)
bzbarsky: review+
Details | Diff | Review
patch v3 - unified diff with function names (3.89 KB, patch)
2009-04-07 11:19 PDT, Michal Novotny (:michal)
cbiesinger: superreview+
Details | Diff | Review

Description zextra 2009-01-26 16:18:38 PST
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.43 Safari/525.19
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5

I have my own little application server. Although the way I have discovered this was due to a bug in my own software, here's a rundown of why do I think this is a violation of standard. Of course, it's not up to me to decide if it's a real bug or a feature.

On every valid HTTP request, my server sent valid HTTP response back to the client (I have Firefox as one of my client browsers).

Example of request URI:

http://localhost:1313/index.php?action=test

Example response:

-----start of response------
HTTP/1.1 200 OK
Content-Length: 37
Connection: close
Content-Type: application/json

test({"volid":"6C9DDEE2","oldsid":"57cabb368034fa07b18e5daf032d1547","newsid":""
,"ver":"1.1"})
-----end of response-------

So, a typical JSON response. But, one might notice that content-length header contains value a lot smaller than actual body size, which is 94 (that's because of a bug in my software, but I don't believe it matters in this case).

Nevertheless, Firefox did grab whole body, irregardless of size specified in content-length. For instance, I have tested same behavior with IE7 and wget, and they both conform to the standard specification (they have grabbed only first 37 bytes of content).

http://www.w3.org/Protocols/HTTP/1.0/spec.html#Content-Length

Because these circumstances are quite rare today, I haven't been able to confirm this behavior anywhere else except in my own lab conditions. But it should be as simple as writing 

Reproducible: Always

Steps to Reproduce:
1. make sure that you have a http server which returns smaller content-length value than size of the body is
2. open up a browser
3. enter exact url leading to mis-generated resource in address bar
4. compare received content size to specified content-length specified in header
Actual Results:  
content body is received in full

Expected Results:  
content body should be received up to the length specified in content-length header
Comment 1 arno renevier 2009-01-27 14:11:14 PST
I cannot reproduce it on linux, neither with 3.0.5, neither with latest trunk.
I've set up a test similar to the one you described (except I use text/plain to make it easier to see in firefox) on http://www.fdn.fr/~arenevier/bugs/mozilla_475444.php On that url, I only see 37 characters.
Comment 2 zextra 2009-01-27 15:24:34 PST
You're right, same happens for me on given url.

Can you try sending exactly the same headers (including content-type and connection: close) as well? It could be related in some way...

Another suggestion might be to try with content length just slightly smaller than real content length (I know that it shouldn't matter, but who knows).

I'll play more with my local app and report back on my findings.
Comment 3 arno renevier 2009-01-27 16:19:42 PST
Hi,
looks like Connection:close is the triggering header:
http://www.fdn.fr/~arenevier/bugs/mozilla_475444-2.php
(I also tested with content-type: application/json, but results are the same)
Comment 4 zextra 2009-01-27 17:28:12 PST
I don't understand if you're able to confirm this by sending connection: close or not?

Anyway, here are few more things. What I noticed when I tested output of your first url via telnet, is that web server stops sending data after specified content-length, and since it's keep-alive connection, it waits for some specified interval, flushes rest of the output, and closes the connection.

When connection: close is sent (second url), whole output is sent at once, thus fulfilling required step number 1. But, even then, I'm still unable to reproduce it using firefox.

For the sake of confirmation, I've exposed my app to public, try testing it using following url:

http://zextra.ath.cx:54321/index.php?action=none

Server response:
------begin-------
HTTP/1.1 200 OK
Content-Length: 88
Connection: close
Content-Type: application/json

none({"volid":"6C9DDEE2","oldsid":"5dcbf65634d694e521069d29e72c95cb","newsid":"","ver":"1.1"})
--------end-------

It still confirms unexpected behavior for me.


Anyhow, I'm not quite sure how important this very special case and deviation from standard really is... What do you think?
Comment 5 Michal Novotny (:michal) 2009-03-17 16:37:14 PDT
This seems to be a feature and not a bug:
http://hg.mozilla.org/mozilla-central/annotate/94678a2f5ae9/netwerk/protocol/http/src/nsHttpTransaction.cpp#l934

But maybe the behavior should be different for HTTP/1.1, see https://bugzilla.mozilla.org/show_bug.cgi?id=83960#c8
Comment 6 Michal Novotny (:michal) 2009-03-17 18:06:45 PDT
Created attachment 367937 [details] [diff] [review]
patch v1

Content-Length header is ignored only in case of HTTP/1.0 when connection isn't persistent.

I tried to write unittest but unfortunately testing httpd doesn't allow to set incorrect Content-Length header. It always computes correct value from bodyInputStream.
Comment 7 Boris Zbarsky [:bz] 2009-03-18 07:04:51 PDT
I think Waldo's working on a patch that would let you set bogus content-lengths...
Comment 8 Boris Zbarsky [:bz] 2009-03-18 07:16:14 PDT
Comment on attachment 367937 [details] [diff] [review]
patch v1

Looks ok, but I'd like Christian to take a look too.
Comment 9 Christian :Biesinger (don't email me, ping me on IRC) 2009-03-18 07:50:39 PDT
Comment on attachment 367937 [details] [diff] [review]
patch v1

It's not like HTTP 1.1 gets this always right, but all cases I've seen the additional data is just garbage (and is generally on keepalive connections anyway. see bug 254181 et al)
Comment 10 Jeff Walden [:Waldo] (remove +bmo to email) 2009-03-18 19:13:21 PDT
(In reply to comment #7)
> I think Waldo's working on a patch that would let you set bogus
> content-lengths...

With bug 396226 that'll be possible, but the most future-proof solution will be in bug 484027, which will let you send whatever random junk you want to the HTTP client, in whatever format you want, and the server won't make any attempt to ensure any formatting whatsoever -- you get complete control except possibly for connection timeouts.  That you can set a bogus Content-Length with only the former is a side effect more than an intended result, and eventually that sort of thing will probably be broken when I implement chunked responses and keepalive connections (the latter of which has already been directly requested, so it's not just a hypothetical feature).
Comment 11 Dão Gottwald [:dao] 2009-04-05 07:55:20 PDT
http://hg.mozilla.org/mozilla-central/rev/905b72e27bb9
Comment 12 Dão Gottwald [:dao] 2009-04-05 08:33:29 PDT
This is causing lots of talos crashes, e.g.
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1238943422.1238945288.3595.gz

Going to backout...
Comment 13 Michal Novotny (:michal) 2009-04-06 17:58:07 PDT
Created attachment 371357 [details] [diff] [review]
patch v2

The problem is that nsHttpTransaction::TakeResponseHead() nulls out mResponseHead. I didn't notice that and it didn't crash on small web pages :-/
Comment 14 Boris Zbarsky [:bz] 2009-04-07 11:04:19 PDT
Comment on attachment 371357 [details] [diff] [review]
patch v2

Michal, you probably want to add the following to your .hgrc to make your hg export patches more readable:

  [diff]
  showfunc = true
  unified = 8

r=me
Comment 15 Michal Novotny (:michal) 2009-04-07 11:19:21 PDT
Created attachment 371486 [details] [diff] [review]
patch v3 - unified diff with function names

Right! I had in my .hgrc

[defaults]
diff=-p -U 8

but this option doesn't take effect for hg export
Comment 17 Ryan VanderMeulen [:RyanVM] 2012-08-16 17:58:25 PDT
https://hg.mozilla.org/mozilla-central/rev/a6c9088eb267

Note You need to log in before you can comment on or make changes to this bug.