937586 - Size column has meaningless value for images and fonts

Status: RESOLVED FIXED

(DevTools :: Netmonitor, defect, P1)

Description

[adam_jar]

Attachment: image1.png

User Agent: Opera/9.80 (Windows NT 5.1) Presto/2.12.388 Version/12.16

Steps to reproduce:

1. Open Dev tools network tab (Ctrl+Shift+Q)
2. Reload web page you are on
3. Check size of any image request


Actual results:

Size of image request is bigger by large margin than actual file(image). For example the biggest image on google home page (as of today) (v1_3fc5b988.png) has 59.4 kB while on network tab the size is 79.19kb. Same thing happens for flash and fonts requests.


Expected results:

Request size in network tab should be equal to uncompressed request content size as it is for html, js and css requests.

Comment 1

[adam_jar]

bug still remains up to firefox 31(latest version). From what I calculated it looks like the difference between what is presented and actual file size is fixed. And it is 25% of actual file size. So net monitor for all uncompressed files(images, flash, fonts) show 125% of the file size instead of 100%( the actual size). 

This bug, when measuring size of web page or its components, renders net monitors pretty much useless

Comment 2

[adam_jar]

correction it's not 25% but 33% (around 1/3 of actual size)

Comment 3

[Markus]

Confirming the issue still exists in Firefox 33. Jpeg files are showing about 25% larger sizes in the network monitor compared to the "Content-Length" header value.

Comment 4

[Aicke Schulz]

Obviously still exists in FF34, because it's not fixed. 

Quite frustrating if you want to optimize the network traffic.

Comment 5

[Aicke Schulz]

I've looked at /browser/devtools/netmonitor/netmonitor-view.js but could not found the line to fix it. But I found informations about data uri and mime, which may be used to calculate a wrong size.

Comment 6

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

I *believe* this is a duplicate of bug 731318, but let's verify once that lands.

The "size" value in the net monitor today is the decoded / decompressed size, not the size on the wire.
After bug 731318, an extra "transferred" column is added to also show the encoded / compressed size (on the wire).

What file size value are you checking to compare to net monitor's value?

Comment 7

[Aicke Schulz]

I compared the size in the size column with the content-length in the headers view and the size on disk. Content-length and size on disk are the same, e.g. ~99kb. But the network size column shows ~130kb. I understand that gzip would reduce size on wire, but we have no gzip for images involved. The size shown is 33% bigger that on disk, for all images! So it's the same as the original poster already mentioned.

Finally, I just checked again what size a base64 encoded image would have (i've got wrong results before) and surprise surprise, it's exactly the size shown in the network size column! Related to op's findings, it seems to be obvious, that all binary files will be shown with their base64 encoded file size.

Comment 8

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Can you retest in a recent Firefox Nightly now that bug 731318 has landed?

Check the new "Transferred" column in the Net Monitor.  It appears to match Content-Length / size on the wire.  Does this give the image size you are expecting for your test case?

I see what you mean about the "Size" column being 33% larger for images.  This seems to be specific to images, because other file types (JS / HTML) appear to show the decompressed size in the "Size" column.  So, there is still something to do to make "Size" more meaningful for images.

Comment 9

[Aicke Schulz]

Attachment: Image Test FF37 Nightly.png

Comment 10

[Aicke Schulz]

1. The nightly alpha crashed several times before working and killed my other firefox session with 100+ Tabs, quite funny :-/

2. As you can see at the attached image above (if you do not want to type http://i.ytimg.com/vi/44iaxwO8xIU/sddefault.jpg), the problem still exists. The size column shows a wrong size, the image is 36kb on disk (if you save it). Transferred and Content-Length seems to be okay.

3. It's not limited to images, e.g. fonts are shown with wrong size, too.

Comment 11

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Aicke Schulz from comment #10)
> 1. The nightly alpha crashed several times before working and killed my
> other firefox session with 100+ Tabs, quite funny :-/

Yes, that's the risk of using Nightly... :( Sorry that it happened.

> 2. As you can see at the attached image above (if you do not want to type
> http://i.ytimg.com/vi/44iaxwO8xIU/sddefault.jpg), the problem still exists.
> The size column shows a wrong size, the image is 36kb on disk (if you save
> it). Transferred and Content-Length seems to be okay.

Right, size is still strange / meaningless, that's "expected" for the moment.  My current guess is that it reflects some kind of in-memory size of the image, but I'll need to look further.

> 3. It's not limited to images, e.g. fonts are shown with wrong size, too.

Yes, I noticed that as well.

Thanks for checking!

Comment 12

[Aicke Schulz]

1. Well, I'll take it as a chance, should have closed them earlier anyway. 

2. Just in case you skipped it, as I mentioned here https://bugzilla.mozilla.org/show_bug.cgi?id=937586#c7 the size is the size of the base64 encoded file.

Comment 15

[Tom Tromey :tromey]

I think this is where the problem starts:

https://dxr.mozilla.org/mozilla-central/source/devtools/shared/webconsole/network-monitor.js#387

This code base64-encodes the response, but none of the users seem to know about this.

Comment 16

[Tom Tromey :tromey]

Attachment: make Size column in network monitor accurate

Comment 17

[Tom Tromey :tromey]

Comment on attachment 8680147 [details] [diff] [review]
make Size column in network monitor accurate

The problem starts in network-monitor.js, which base64-encodes the response
text.

However, just removing this code causes problems elsewhere, most notably
in netmonitor-view.js, which assumes that the "encoding" property can be
used to construct a data: URL.

So, this patch takes a slightly different approach, namely changing
addResponseContent to pass through the original size.

I also noticed that the "Encoding" field in the image response viewer is
always "base64".  I couldn't find any other spot that can set this to anything
else.  So, since it doesn't seem useful, this patch removes it entirely.

No new test because there is an existing test that exercises exactly this
functionality -- it was just incorrect.

Comment 18

[YF (Yang)]

(In reply to Tom Tromey :tromey from comment #17)
> Comment on attachment 8680147 [details] [diff] [review]
> make Size column in network monitor accurate
> 
> I also noticed that the "Encoding" field in the image response viewer is
> always "base64".  I couldn't find any other spot that can set this to
> anything
> else.  So, since it doesn't seem useful, this patch removes it entirely.

Open the https://developer.cdn.mozilla.net/static/img/logo_sprite.7d36c4a1422b.svg, the "Encoding" field show as undefined.

While it may be just a bug. No other possible values for "response.encoding".

Comment 19

[Tom Tromey :tromey]

Comment on attachment 8680147 [details] [diff] [review]
make Size column in network monitor accurate

Clearing the review while I investigate comment #18.
While I don't see the "Encoding" field at all (the patch removed it),
the image thumbnail doesn't display, so I think something else is wrong.

Comment 20

[Brian Grinstead [:bgrins]]

(In reply to Tom Tromey :tromey from comment #19)
> Comment on attachment 8680147 [details] [diff] [review]
> make Size column in network monitor accurate
> 
> Clearing the review while I investigate comment #18.
> While I don't see the "Encoding" field at all (the patch removed it),
> the image thumbnail doesn't display, so I think something else is wrong.

I don't see the image showing up as the favicon for the tab, either.  And the preview doesn't display in Dev Edition (I see a white square that says 'cannot load image' when I hover it).  So this may be an existing issue.

Comment 21

[Tom Tromey :tromey]

Comment on attachment 8680147 [details] [diff] [review]
make Size column in network monitor accurate

It's a pre-existing issue.
I'll file another bug.

Comment 22

[Tom Tromey :tromey]

Bug 1219771.

Comment 23

[Tom Tromey :tromey]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c6c5de9cd934

Comment 24

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/49cfa28ef596

Comment 25

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/49cfa28ef596