597918 - Incessant http get requests when Javascript animates same image in two tabs

Status: NEW

(Core :: Graphics: ImageLib, defect)

Description

[David Brodrick]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.14eol) Gecko/20070505 (Debian-1.8.0.15~pre080614i-0etch1) Galeon/2.0.2 (Debian package 2.0.2-4)
Build Identifier: Firefox versions 3.0 to 4.0 at least.

I use Javascript to animate images on my website by using a timer to update the src field of an image. I have noticed in my webserver logs that Firefox browsers can sometimes get into a state where they issue a new http get request to the server for every image on every iteration through the animation loop.

This results in a flood of unnecessary traffic between the browser and the server, consisting of if-modified-since get requests and 304 not-modified responses. As well as creating excess traffic this also degrades the user experience as the animation becomes dependent on the slow I/O of communicating with the server.

I have now been able to isolate and reproduce the problem. It happens when two tabs (or presumably windows) which are using Javascript to make the animations both refer to the same images on the server (for instance if you load the same page in two different tabs). The two instances seem to behave properly when first loaded, but as soon as you hit reload in one of the tabs the flood of traffic begins with 100% reproducibility.

I have tried this on everything from Galeon 2.0.2 on Linux to Firefox 4.0b6 on Windows XP, and the problem is reproducible in all cases.

Reproducible: Always

Steps to Reproduce:
The problem can be demonstrated by loading the supplied URL in two different tabs and then hitting F5 to reload one of the tabs.
Actual Results:  
If you take the steps above to reproduce the problem and then monitor the traffic between the browser and server you will see an endless stream of get (if-modified-since) requests and 304 (not-modified) responses.

Expected Results:  
The normal and expected behaviour is to issue a get for each image once only, when the page is first loaded, and then simply refer to the cached copy for the remainder of the animation. This is what all other browsers do and this is what Gecko does unless the image is referred to by two different tabs as discussed.

The problem is only evident when multiple pages with javascript animations refer to the same images on the server. Without knowing anything about the actual implementation, it seems the issue is associated with having multiple threads trying to access the same cached image. I am not sure why it seems okay when the tabs are first loaded and only triggers once one of the tabs reloads.

In the case of my server the pages automatically reload every 10 minutes, so clients end up in this state quite quickly.

I am taking a punt that the Javascript engine is reponsible, please assign this to the appropriate component, such as the cache management machinery, if I am incorrect. Many thanks.

Comment 1

[Boris Zbarsky [:bzbarsky]]

What exact headers are sent with the response in this case for the 304 response?  The 200 response sends a Last-Modified that's 2 hours ago and no Expires or max-age, so we compute an expiration time of 12 minutes for it.  After 12 minutes pass, we will make another GET instead of hitting the cache, and get the 304 response... and update the expiration information with whatever information that 304 returns.  What does it return?

Comment 2

[David Brodrick]

Many thanks for your response. As part of debugging this problem I tried configuring the apache expires module so that the images that are normally animated in operational use of my website have the following headers set:

Cache-Control: max-age=86400
Expires: Tue, 21 Sep 2010 02:01:43 GMT
(ie expires 24 hours from now)

However this made no difference to the fault. I didn't think to configure the cache-control header for the simple test case I put together for the bug report because it is incidental to the actual problem.

The get request happens every time the animation changes to a new images, ie five times per seond for the demonstration I put together. Not every 12 minutes.

I can see in the server logs that you only loaded the page once. Please try loading it in two different tabs and then hit reload in one of them in order to trigger the bug.

Comment 3

[David Brodrick]

Here is an abbreviated excerpt from my server logs. It actually seems a new get request is issued every time the animation frame is updated in either tab. Ie for the bugdemo.html page I put together, there is 10 (ten) get requests to the server every second if you have two tabs open.

[20/Sep/2010:11:27:14 +1000] "GET /temp/white.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/black.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/black.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/white.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/white.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/black.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/black.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/white.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/white.jpg HTTP/1.1" 304
[20/Sep/2010:11:27:14 +1000] "GET /temp/black.jpg HTTP/1.1" 304

I hope to convince you that there is a real problem and this is not just the normal cache behaviour.

Comment 4

[Boris Zbarsky [:bzbarsky]]

I loaded the page once, then got each of the images via wget, actually.  And again, what happens after the first 304 depends on the headers sent with the 304 response; it may not take 12 minutes to redo a GET at that point.

But this is still an imagelib issue, not a JS engine one; don't move it back there.  If the above cache-control header was also sent with the 304, then chances are this is the imagelib cache per-document pinning suddenly missing for some reason after a reload.

I believe you there's a real problem; I'm just trying to diagnose what might trigger it, and all the data I can get is helpful.  If you have the exact headers you send with your 304 available offhand, that would save Joe or Bobby the time needed to do an HTTP log to get those headers...

Comment 5

[David Brodrick]

Below is the full 304 response headers obtained with the LiveHTTPHeaders plugin while the browser was in the broken state constantly polling the server:

HTTP/1.1 304 Not Modified
Date: Mon, 20 Sep 2010 02:31:26 GMT
Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9 with Suhosin-Patch
Connection: Keep-Alive
Keep-Alive: timeout=15, max=92
Etag: "144023-1ce3-490a5852a7b40"

Thanks.

Comment 6

[Boris Zbarsky [:bzbarsky]]

That doesn't update the expiration time, right? So once the original 200 response expires it will stay expired.... 

That said, it sounds like the two documents thing is still pointing out a separate issue, since the problem there starts as soon as you reload, not when the image actually expires... but the reload likely causes us to revalidate and get that 304 response, which says that the image should be considered expired thereafter.

It's worth testing whether sending a useful Expires or max-age with the 304 helps, since that will narrow down the set of possibly subsystems involved here (http cache vs image cache).

Comment 7

[David Brodrick]

Okay now I see what you are looking for. I should have been more exhaustive:

The 304 response above is for the testcase images. Below is the 304 header for the actual image directory my website normally uses. In this case the expiration and max-age headers do get updated each time:

HTTP/1.1 304 Not Modified
Date: Mon, 20 Sep 2010 03:59:01 GMT
Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9 with Suhosin-Patch
Connection: Keep-Alive
Keep-Alive: timeout=15, max=85
Etag: "ded094-2cea-490a853762600"
Expires: Tue, 21 Sep 2010 03:59:01 GMT
Cache-Control: max-age=86400

So having the 304 header update the expiration time does not change the behaviour of the browser (given that I see the problem I am reporting in normal operation use using the image directory that posts the additional headers).

Comment 8

[Boris Zbarsky [:bzbarsky]]

Great.  If you can make the testcase send headers like that too, just so the debugging doesn't run into that issue, that would be awesome.  Joe, Bobby, Alon can one of you look at this?

Comment 9

[David Brodrick]

Done. The testcase now includes the Expires and Cache-Control: max-age headers in the 304 response.

Comment 10

[David Brodrick]

Hi Folks,
Do you still need me to keep the testcase files in that temp directory on my site?
Thanks,
  David

Comment 11

[Boris Zbarsky [:bzbarsky]]

David, if you can keep them up for now, that would be great.  I think the imagelib folks have been swamped with 4.0 blocker bugs so far.  :(

Comment 12

[David Brodrick]

Hi,
I note this bug is still present in Firefox 23.0.

I am sure if would be an easy fix for somebody familiar with the relevant code.
Regards,
  Dave