Firebug + firefox net console dont reliably report response headers

RESOLVED INCOMPLETE

Status

DevTools
Netmonitor
RESOLVED INCOMPLETE
4 years ago
9 days ago

People

(Reporter: c3172354, Unassigned)

Tracking

34 Branch
x86_64
Windows 8.1

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

4 years ago
User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0
Build ID: 20141125180439

Steps to reproduce:

I spent a large amount of time thinking that my tomcat 7 setups weren't compressing things correctly

http://stackoverflow.com/questions/27816958/problems-getting-tomcat-7-to-use-gzip-compression

I was trying to check that gzip compression was in effect on our servers using the different firefox tools available (the normal net console, firebug's net console and yslow).  I had firebug installed just because yslow requires it, but both consoles were failing in a similar way.


Actual results:

It appeared that gzip compression was not working when in fact it was.  This was also intermittent and led to me spending somewhere in the region of 2 days of googling about tomcat gzip compression, diffing and changing config options, and generally being baffled about what could be causing what I was seeing.

When looking at a compressed response firefox was intermittently misreporting what was going on and showing a type of bytes and a content length with a number, rather than 'chunked' and type gzip.

Some of the time it was reporting the right thing which really confused things more as it wasn't giving me a clear picture of the effect (if any) that the changes I was making were having to the state of the compression.

After going round in circles I accidentally used chrome on a computer which didn't have firefox which correctly reported that compression was indeed switched on in the place where it didn't seem to be working at all according to firefox (see the stack overflow post for more details).  After seeing that I then began checking what was going on with curl and wireshark.

Curl was consistently reporting compression as being on and then wireshark confirmed that compression was indeed on for a browser request that firefox reported it as not being.

I could see this in two ways - the headers shown in wireshark were different (showing gzip and chunked as you'd expect) and because you could look at the content and see that it was compressed as it looked like gibberish.

In firefox you can't see the raw content (which is fine) but the headers that were reported were incorrect - they did not say gzip and specified an exact length (which is what you would see if the file was not compressed).  


Expected results:

Firefox network console should report the response headers from the server correctly.  It clearly doesn't do this as I observed from the discrepancy between the sniffed packets in wireshark and what was reported by the developer tools.

I dont know what is going on there - possibly those values have been calculated after decompression in an effort to be more helpful.  If so it is definitely not more helpful!!  Please report exactly the headers that the server sends back!
(Reporter)

Comment 1

4 years ago
If you want to contact me further about this then please post on the stackoverflow answer I've linked to.  I signed up for this with a temporary email address as I dont want my email address to be public + receive a load of spam so wont receive any email notifications or whatever you might normally get

Comment 2

4 years ago
Victor, can you suggest what to do here to figure out what's going on?
Component: Untriaged → Developer Tools: Netmonitor
Flags: needinfo?(vporof)
Should probably be fixed by bug 731318. Check out Firefox Nightly.
Flags: needinfo?(vporof)
Status: UNCONFIRMED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 731318
(Reporter)

Comment 5

4 years ago
Created attachment 8545881 [details]
firefox headers.PNG

Here is what the firefox debugger shows...
(Reporter)

Comment 6

4 years ago
Created attachment 8545882 [details]
firefox raw headers view.PNG

Here is what the raw headers view shows (I believe this is the new bit?)...
(Reporter)

Comment 7

4 years ago
Here is the request/response as observed by wireshark

GET /Veolia/concat.js?r=20150106-1145 HTTP/1.1
Host: trac-leeds-11
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://trac-leeds-11/Veolia/
Cookie: JSESSIONID=68C458112C46D64A0DF13A27FEBFC564; ext-mainEventsGridState=o%3Acolumns%3Da%253Ao%25253Aid%25253Ds%2525253Ah1%255Eo%25253Aid%25253Ds%2525253Ah2%255Eo%25253Aid%25253Ds%2525253Ah3%255Eo%25253Aid%25253Ds%2525253Ah4%255Eo%25253Aid%25253Ds%2525253Ah5%255Eo%25253Aid%25253Ds%2525253Ah6%255Eo%25253Aid%25253Ds%2525253Ah7%255Eo%25253Aid%25253Ds%2525253Ah8%255Eo%25253Aid%25253Ds%2525253Ah9%255Eo%25253Aid%25253Ds%2525253Ah10%255Eo%25253Aid%25253Ds%2525253Ah11%255Eo%25253Aid%25253Ds%2525253Ah12%255Eo%25253Aid%25253Ds%2525253Ah13%255Eo%25253Aid%25253Ds%2525253Ah14%255Eo%25253Aid%25253Ds%2525253Ah15%255Eo%25253Aid%25253Ds%2525253Ah16%255Eo%25253Aid%25253Ds%2525253Ah17%255Eo%25253Aid%25253Ds%2525253Ah18%255Eo%25253Aid%25253Ds%2525253Ah19
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Accept-Ranges: bytes
ETag: W/"29625-1420633264500"
Last-Modified: Wed, 07 Jan 2015 12:21:04 GMT
Content-Type: application/javascript;charset=UTF-8
Transfer-Encoding: chunked
Content-Encoding: gzip
Vary: Accept-Encoding
Date: Thu, 08 Jan 2015 12:17:58 GMT

a
..........
200
.}.w....W.yw].Fd....s..>..7n.8..t.\..X.$6....m]Y.}g. ......g..IE....0..?.?..


As you can see it is actually compressed and firefox is still getting the headers completely wrong.
Status: RESOLVED → UNCONFIRMED
Resolution: DUPLICATE → ---
(Reporter)

Comment 8

4 years ago
The last 3 comments are having just tried it on the nightly build 37.0a1 (2015-01-07).

It's still not working so I've added a couple of screen shots of what firefox says along with what wireshark says.

I guess the new bit is the raw headers?  I've not noticed that before anyway, either way its still getting it wrong.  

I don't know the code at all but surely nothing should be messing around with the values in here - it should just be a matter of reading them off the wire and then just displaying them straight. It certainly seems like this isn't happening and that they are tampered with on the way through.
Are you trying Firefox Nightly in e10s mode?  Please try disabling e10s for now (in the Preferences), as not all parts of Net Monitor work correctly in e10s.  See bug 982319 for more info on this.

The part that is new is the "Transferred" column, which correctly matches in size on the wire.  In non-e10s mode, the Transferred and Size columns give different values, where Transferred is size on wire, and Size is the uncompressed size.

However, I see what you mean that some headers have shifted:

* Net Monitor is missing Transfer-Encoding
* Net Monitor is missing Content-Encoding
* Net Monitor added Content-Length

Can you re-test in Nightly with e10s disabled to see if you get different results?
Flags: needinfo?(c3172354)
(Reporter)

Comment 10

4 years ago
Hi,

I've just checked with it definitely not in e10s mode and see the same thing.

I dont think I was in e10s mode before anyway - I dont remember clicking to enable it.

Cheers
Flags: needinfo?(c3172354)
Okay, I am not able to reproduce this behavior on just any site using these headers, so something must be specific to your site.

Is there a public URL where I can test this same request?  Or if not, could you attach a Wireshark / Fiddler / something capture that can easily be replayed?
Flags: needinfo?(c3172354)
(Reporter)

Comment 12

3 years ago
Hi, Sorry didn't realise there had been more activity here.  So there's no public url + I'm not on the project anymore.
I'm not sure what else I can do to help.  Thanks for taking an interest though!
Flags: needinfo?(c3172354)

Updated

3 years ago
Status: UNCONFIRMED → RESOLVED
Last Resolved: 4 years ago3 years ago
Resolution: --- → INCOMPLETE

Updated

9 days ago
Product: Firefox → DevTools
You need to log in before you can comment on or make changes to this bug.