101165 - GETing parts of page twice

Status: RESOLVED DUPLICATE of bug 92722

(Core :: Networking: Cache, defect)

Description

[Jeremy M. Dolan]

http://news.bbc.co.uk/ has a total of 48 items on it (html, css, gif, jpeg). I
emptied my cache, disabled disk cache, and restarted. I traced the network as
mozilla opened http://news.bbc.co.uk/. Two of the gif files were requested twice.

/ is request 1.
/furniture/dotted_line.gif is request 8 AND 11.
/furniture/nav/ukfs/languages.gif is request 9 AND 17.

Not sure if this is a cache problem, or if something else should be picking it off.

Comment 1

[Jeremy M. Dolan]

Correction, on that first test, THREE gifs were requested twice. So there are 47
objects only on that page. This time, Mozilla sent 53 GETs. 4 files were
requested twice, and /furniture/grey_pixel.gif was requested THREE times, two of
those three requests were within a few tenths of a second, so it's not as if it
gave up on the first attempt (I'm on a decently fast connection, cable).

Comment 2

[gordon]

If you disabled the disk cache, HTTP would try to use the memory cache. 
However, ImgLib is also using the memory cache, which is by default set to 4 Mb.

I set my disk cache capacity to zero, increased my memory cache capacity to 20
Mb, and visited the site.  The memory cache filled with about 9 Mb of data (from
both HTTP, and ImgLib - including chrome images).  If your memory cache is set
less than 9 Mb for that page, and you have disabled the disk cache, I would
expect multiple GET requests to be made.

Comment 3

[Jeremy M. Dolan]

Hate to disagree, but...

My cache previously was 8M (8*1024K). I set it to 16*1024, restarted, and again,
3 images were requested twice. I set it to 32*1024, restarted, and AGAIN 3
images were requested twice.

I checked about:cache and it's definatly not full:

Number of entries: 139
Maximum storage size: 33554432 Bytes
Storage in use: 1178256 Bytes

I'm going to do one more dump, and make sure the multiple GETs I'm seeing aren't
due to TCP retransmits, which is the only other thing I can think of.

> from both HTTP, and ImgLib - including chrome images

if internal browser stuff is consuming the same cache, the description definatly
should mention that. It only says the cache is used for "web pages", which will
lead to people like myself not factoring in room for that. If I set mem cache to
8M, i expect 8M worth of web page html and img's to be cached, not for the
browser to suck up half that for it's UI.

Comment 4

[Jeremy M. Dolan]

OK, this is definatly happening.

Here's two TCP connections that happened. All I did was start up moz (32M memory
cache), then go straight to the url (bbc news).



GET /furniture/top_bbc_logo.gif HTTP/1.1
Host: news.bbc.co.uk
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.4+) Gecko/20010922
Accept: text/xml, application/xml, application/xhtml+xml, text/html;q=0.9,
image/png, image/jpeg, image/gif;q=0.2, text/plain;q=0.8, text/css, */*;q=0.1
Accept-Language: en-us 
Accept-Encoding: gzip, deflate, compress;q=0.9
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Connection: keep-alive
Cookie: SITESERVER=ID=6ab15483aa310e84194cbde837312024
Referer: http://news.bbc.co.uk/

HTTP/1.1 200 OK
Date: Mon, 24 Sep 2001 23:25:50 GMT
Server: Apache/1.3.14 (Unix)
Last-Modified: Tue, 07 Aug 2001 16:51:00 GMT
ETag: "40938b-be-3b701c74"
Accept-Ranges: bytes
Content-Length: 190
Connection: close
Content-Type: image/gif

GIF89a/data blah blah
blah
blah








GET /furniture/top_bbc_logo.gif HTTP/1.1
Host: news.bbc.co.uk
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.4+) Gecko/20010922
Accept: text/xml, application/xml, application/xhtml+xml, text/html;q=0.9,
image/png, image/jpeg, image/gif;q=0.2, text/plain;q=0.8, text/css, */*;q=0.1
Accept-Language: en-us
Accept-Encoding: gzip, deflate, compress;q=0.9
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Connection: keep-alive
Cookie: SITESERVER=ID=6ab15483aa310e84194cbde837312024
Referer: http://news.bbc.co.uk/

HTTP/1.1 200 OK
Date: Mon, 24 Sep 2001 23:25:47 GMT
Server: Apache/1.3.14 (Unix)
Last-Modified: Tue, 07 Aug 2001 16:51:00 GMT
ETag: "40938b-be-3b701c74"
Accept-Ranges: bytes
Content-Length: 190
Connection: close
Content-Type: image/gif

GIF89a/dckhfsdkghdhgdfg
dfg
dfg









Notice there are 3 seconds between the requests.
Notice mozilla asked for keepalive, bbc denied.

Here's the order of GETs, that time:

  GET / 
  GET /stylesheets/corenews.css 
  GET /furniture/top_bbc_logo.gif 
  GET /furniture/nothing.gif 
  GET /furniture/nav/lhs_sitepuffs/news_event.gif 
  GET /furniture/top_bbc_logo.gif 
  GET /furniture/nothing.gif 
  GET /furniture/nav/lhs_sitepuffs/news_event.gif 
  GET /furniture/nav/lhs_sitepuffs/weather.gif 
  GET /furniture/dotted_line.gif 
  GET /furniture/nav/ukfs/languages.gif 


You can see three duplicate files in there...
/furniture/nav/lhs_sitepuffs/news_event.gif
/furniture/nothing.gif
/furniture/top_bbc_logo.gif

Comment 5

[Jeremy M. Dolan]

Err, I pasted those in backwards, :47 then :50, obviously.

Note the data is the same in both, i just typed in some stuff there, since it's
binary in the original copy. Another clarification, that GET list is cut off...
there were 50 (for 47 objects), I just pasted the top as thats where all the
dupes happened to be, this time. Sometimes they happen lower, even right by the
bottom. Seems fairly random.

Comment 6

[Jeremy M. Dolan]

Here's an interesting thing... those files are listed twice in
about:cache?device=mem, with differant sizes:

           Key: http://news.bbc.co.uk/furniture/top_bbc_logo.gif
     Data size: 1520 Bytes
   Fetch count: 1
 Last Modified: Mon Sep 24 18:23:05 2001
       Expires: Sat Sep 29 14:14:34 2001

           Key: http://news.bbc.co.uk/furniture/top_bbc_logo.gif
     Data size: 190 Bytes
   Fetch count: 1
 Last Modified: Mon Sep 24 18:23:05 2001
       Expires: Sat Sep 29 14:14:34 2001

           Key: http://news.bbc.co.uk/furniture/nothing.gif
     Data size: 8 Bytes
   Fetch count: 45
 Last Modified: Mon Sep 24 18:23:05 2001
       Expires: Sun Sep 30 09:44:36 2001

           Key: http://news.bbc.co.uk/furniture/nothing.gif
     Data size: 43 Bytes
   Fetch count: 1
 Last Modified: Mon Sep 24 18:23:05 2001
       Expires: Sun Sep 30 09:44:36 2001

Have fun with this one, bub. :-)

Comment 7

[gordon]

The reason the files are listed twice is because one of the listings in
about:cache is for HTTP and the other is for ImgLib (which uses very similar
looking keys).  ImgLib stores the decoded image. If you click on the cache
entry, you will get an expanded view that contains more information about it.

I misread my "Storage in use" figure. It wasn't 9 Mb, but rather just short of 1
Mb  (man, I'd really like to put the appropriate localized number breaks in there).

Darin, any ideas why we would be requesting three of these files more than once?

Comment 8

[Darin Fisher]

pavlov could help with this... are images requested multiple times only when the
user presses reload or shift-reload?

Comment 9

[gordon]

Hopefully, I'll get some attention from Pav in the 0.9.6 timeframe.

Comment 10

[gordon]

Targeting for necko performance milestone 0.9.7.

Comment 11

[gordon]

This looks like bug 92722.  Marking duplicate.

*** This bug has been marked as a duplicate of 92722 ***