Open Bug 583351 Opened 10 years ago Updated 14 days ago

Firefox sends GET request twice to server for a dynamically-generated small PNG image

Categories

(Core :: ImageLib, defect)

x86_64
Windows 7
defect
Not set

Tracking

()

People

(Reporter: gbegen, Unassigned)

References

(Blocks 1 open bug)

Details

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8

We have a server that can serve up dynamically generated PNG images based on information provided in the GET URL. If we navigate to one of these URLs in Firefox 3.6.8 on Windows 7 64-bit, Firefox makes two requests to the server every time the browser is refreshed.

The behavior seems to be dependent upon the size of the PNG image returned. A small, 627x43 pixel, image exhibits the problem. A larger, 1100x600 pixel, image does not. I have not narrowed down the exact size at which the problem occurs.

If that generated file is placed directly in the file system to be served by IIS in a non-dynamic fashion, Firefox only makes one request per refresh.

Without changing the server, if the request comes from FireFox 3.6.8 running on an XP 32-bit machine, only one request per refresh is made.

Reproducible: Always

Steps to Reproduce:
1. Create a server that dynamically generates a 627x43 pixel PNG image returned as content-type=image/png.
2. Navigate to the URL for that image from the server.
3. Press refresh
Actual Results:  
By reviewing server logs or using a tool like Live HTTP Headers add-on, notice that each refresh makes two GET requests to the server.

Expected Results:  
Each refresh should only have created a single request to the server.

Log from Live HTTP Headers add-on:

http://localhost/Label/label.ashx/label-200.png?AgAAAMDqrjQn_MwIY-7OukKOuBr2VyNYKWYerXXss_B4nONLtWIQ0GCY93fj3jjXxp8apxymC_IK_Geo077SlOg271ZebePG8rt8BmZGwQZgYGRgYMIdmldSlJicnZiUk8oENIAjzNMv0ifYMfL___8ihgZmlgYKzv4BAa5BCj6O3q4KLkH__7MEufr5M_mFsVpYmhoZspgaWRpzWRiYmhqYmBkZGjEDDeH3cg0O9nR2VHAKDXJ3DQkBmiVlYW5sruDm4xri7AE0LMA7PFLBMSBEwdTcACjL7uOo4Osa7Mjk7MhqaWhpYsRibGRs8f8_b2heZklqikJwSWJJavH__xwMHAwQaxk-MiADJgbGSFbGf_8ZGEDeu8LOwCBgs4eRIYuToWybEeOleEaIsmQw-cKqlsGAgUOAL7DGdYJnTlTngbcibY8Xs7aeOSvUNcvrmuRHL93IlY_WuytuaX6-f8ohBjbH3MSq_DwGBof0U6a_z3BA6Q4GQoCNgWaAhQlIQLwFAGJWdDE=

GET /Label/label.ashx/label-200.png?AgAAAMDqrjQn_MwIY-7OukKOuBr2VyNYKWYerXXss_B4nONLtWIQ0GCY93fj3jjXxp8apxymC_IK_Geo077SlOg271ZebePG8rt8BmZGwQZgYGRgYMIdmldSlJicnZiUk8oENIAjzNMv0ifYMfL___8ihgZmlgYKzv4BAa5BCj6O3q4KLkH__7MEufr5M_mFsVpYmhoZspgaWRpzWRiYmhqYmBkZGjEDDeH3cg0O9nR2VHAKDXJ3DQkBmiVlYW5sruDm4xri7AE0LMA7PFLBMSBEwdTcACjL7uOo4Osa7Mjk7MhqaWhpYsRibGRs8f8_b2heZklqikJwSWJJavH__xwMHAwQaxk-MiADJgbGSFbGf_8ZGEDeu8LOwCBgs4eRIYuToWybEeOleEaIsmQw-cKqlsGAgUOAL7DGdYJnTlTngbcibY8Xs7aeOSvUNcvrmuRHL93IlY_WuytuaX6-f8ohBjbH3MSq_DwGBof0U6a_z3BA6Q4GQoCNgWaAhQlIQLwFAGJWdDE= HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Cache-Control: max-age=0

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: image/png
Server: Microsoft-IIS/7.5
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Fri, 30 Jul 2010 21:07:43 GMT
Content-Length: 4011
----------------------------------------------------------
http://localhost/Label/label.ashx/label-200.png?AgAAAMDqrjQn_MwIY-7OukKOuBr2VyNYKWYerXXss_B4nONLtWIQ0GCY93fj3jjXxp8apxymC_IK_Geo077SlOg271ZebePG8rt8BmZGwQZgYGRgYMIdmldSlJicnZiUk8oENIAjzNMv0ifYMfL___8ihgZmlgYKzv4BAa5BCj6O3q4KLkH__7MEufr5M_mFsVpYmhoZspgaWRpzWRiYmhqYmBkZGjEDDeH3cg0O9nR2VHAKDXJ3DQkBmiVlYW5sruDm4xri7AE0LMA7PFLBMSBEwdTcACjL7uOo4Osa7Mjk7MhqaWhpYsRibGRs8f8_b2heZklqikJwSWJJavH__xwMHAwQaxk-MiADJgbGSFbGf_8ZGEDeu8LOwCBgs4eRIYuToWybEeOleEaIsmQw-cKqlsGAgUOAL7DGdYJnTlTngbcibY8Xs7aeOSvUNcvrmuRHL93IlY_WuytuaX6-f8ohBjbH3MSq_DwGBof0U6a_z3BA6Q4GQoCNgWaAhQlIQLwFAGJWdDE=

GET /Label/label.ashx/label-200.png?AgAAAMDqrjQn_MwIY-7OukKOuBr2VyNYKWYerXXss_B4nONLtWIQ0GCY93fj3jjXxp8apxymC_IK_Geo077SlOg271ZebePG8rt8BmZGwQZgYGRgYMIdmldSlJicnZiUk8oENIAjzNMv0ifYMfL___8ihgZmlgYKzv4BAa5BCj6O3q4KLkH__7MEufr5M_mFsVpYmhoZspgaWRpzWRiYmhqYmBkZGjEDDeH3cg0O9nR2VHAKDXJ3DQkBmiVlYW5sruDm4xri7AE0LMA7PFLBMSBEwdTcACjL7uOo4Osa7Mjk7MhqaWhpYsRibGRs8f8_b2heZklqikJwSWJJavH__xwMHAwQaxk-MiADJgbGSFbGf_8ZGEDeu8LOwCBgs4eRIYuToWybEeOleEaIsmQw-cKqlsGAgUOAL7DGdYJnTlTngbcibY8Xs7aeOSvUNcvrmuRHL93IlY_WuytuaX6-f8ohBjbH3MSq_DwGBof0U6a_z3BA6Q4GQoCNgWaAhQlIQLwFAGJWdDE= HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: image/png
Server: Microsoft-IIS/7.5
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Fri, 30 Jul 2010 21:07:43 GMT
Content-Length: 4011
----------------------------------------------------------
Do you get this without addons -> http://support.mozilla.com/en-US/kb/Safe+Mode
Yes, this occurs with all add-ons disabled.

Based on your link, I also tried safe-mode with the same results.
Do you only load the png directly or do you have a html file that includes this image ?
This only happens when navigating directly to the PNG. If we create an HTML page with an <img> tag that refers to the image, it is only retrieved once.
Component: General → ImageLib
Product: Firefox → Core
QA Contact: general → imagelib
Joe, this rings some bells about bugs you've fixed, but my memory is pretty fuzzy. Anything come to mind?
Presumably some sort of failure in coalescing things wrt LoadImageWithChannel?  In particular, perhaps not coalescing the load of the <img> in the image document with the LoadImageWithChannel load?
I wonder if it's the same thing as bug 552605...
There doesn't seem to be any redirect involved here, no.
Geoff, if you could set up a public server that has such a configuration, that'd help a lot. I don't have access to IIS, or I'd do it myself.
(And if this doesn't happen anymore, please let us know!)
This issue is still exists even in Firefox 9.0.1. 

We do get two calls to server for the dynamic source. 

our element is like this 

<img height="1px" width="1px" src="http://example.com/view?param=value" />

Somehow we get one call if there is not much html elements on the page! We get two calls to backend if there are some other elements on the page, even commented code or a simple div element causing double call..
The number of elements on the page just affects whether there's a prefetch or not, I would think.

The real question is why you're getting an image cache miss...  Hard to tell without a testcase.  Is there one you can make public?
You don't have multiple references to this URL, right?
This appears to be an issue with the taskbar previews in windows 7.  My details:

Win 7 x64.
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20100101 Firefox/11.0

I'm guessing this is related to the old url-bar icon issue from bug 304574.  I ran some tests using a PHP script that generated an image with a number counter for each image.  I noticed that while the window, url-bar, and tab icons all matched, the icon in the taskbar did not.  The incorrect icon is only visible if "Options -> Tabs -> Show tab previews in windows taskbar" is enabled, but the double-request appears to happen regardless of the value of this setting.

My test script can be accessed at http://linode.aoeex.com/imglog.php 
It outputs a log of requests at http://linode.aoeex.com/log.html
Image showing the affected icon: http://linode.aoeex.com/different.png
A few additional findings:

The addition no-cache headers such as those output by php's session_start() appear to prevent this second request.

The addition of a Last-Modified: header with a date at least 10 seconds old will also prevent the second request, provided there are no query-string parameters in the url.  If a query string is present it seems to go ahead with the second request regardless of the last-modified header's value.
Duplicate of this bug: 830381
I can confirm that the image is rquested twice with the url from bug 830381.
2 http GET requests are send, one normal load and one with if-modified-since header.
Status: UNCONFIRMED → NEW
Ever confirmed: true
i can confirm this bug for 23.0.1. Adding a cache-control max-age=1 header does seem to prevent the second request, but my guess is that is is just because the second request is "answered" by the cache.

Interestingly, the second request does not show up in the "network" profiler (Ctrl+Shift+Q), neither can it be seen in firebug.
Confirm for 25.0.1 on win7 64 bits, for a .gif image which is in fact a "tracking-pixel" beacon and thus has to be delivered with no-cache headers.

2nd request does not show up in firebug.
I can't believe that this bug is still around
It is still present in Firefox 27
Is it really that hard to fix this?
I am not sure, maybe I do not understand something. Maybe my web server is not implemented correctly: maybe the first time it "asks" server whether image expired or not;
Like I said, I am not sure, but for now I think that this is rather Firefox bug
I open image in separate tab, then I tap F5, then it loads the image twice. WTF???
> Is it really that hard to fix this?

Yes, because no one can reproduce it so far.

If you have a testcase that shows the problem, please link to it.
(In reply to Boris Zbarsky [:bz] (reviews will be slow; ask someone else) from comment #21)
> > Is it really that hard to fix this?
> 
> Yes, because no one can reproduce it so far.

#!/usr/bin/env ruby
require 'base64'
require 'webrick'

server = WEBrick::HTTPServer.new :Port => 8000

# Request sent twice because no Expires header (?)
server.mount_proc '/bad/image.gif' do |req, res|
  res['Content-Type'] = 'image/gif'
  res.body = Base64.decode64('R0lGODlhAQABAIAAAPj8/wAAACwAAAAAAQABAAACAkQBADs=')
end

# Request sent once
server.mount_proc '/good/image.gif' do |req, res|
  res['Content-Type'] = 'image/gif'
  res['Expires'] = 'Wed, 12 Mar 2050 17:49:57 GMT'
  res.body = Base64.decode64('R0lGODlhAQABAIAAAPj8/wAAACwAAAAAAQABAAACAkQBADs=')
end

server.start
OK, and what steps to reproduce running against that server?
(In reply to Boris Zbarsky [:bz] (reviews will be slow; ask someone else) from comment #23)
> OK, and what steps to reproduce running against that server?

With the server running in a terminal:

When accessing http://127.0.0.1:8000/bad/image.gif webrick shows it receives two requests.

http://127.0.0.1:8000/good/image.gif seems to work fine.
Thanks.

So if I do that, I get a second request to imagelib for this URI:

  http://127.0.0.1:8000/bad/image.gif#-moz-resolution=32,32

This is coming from the Firefox favicon for the tab.

The extra fragment identifier causes us to miss the image cache (because that keys on the whole URI), and the fact that the image has no HTTP freshness information means we end up missing the HTTP cache as well (since the image is considered stale).

Does it really make sense to do the bug 828508 bits for cases when we're using the main-page image as favicon?  If not, worth filing a bug blocking this one on that and retesting once that's fixed...
Flags: needinfo?(jfkthame)
Flags: needinfo?(dao)
Alternately, should the image cache ignore the fragment identifier?  It's not clear to me that this is safe, in general...
Flags: needinfo?(seth)
The image cache cannot ignore the fragment identifier, because the image cache actually caches the decoded version of the image, and the decoded version varies depending on the fragment identifier.

The real cause is that RasterImage cannot hold more than one decoded version of an image. I cannot tell you how many problems this is causing. I just recently started looking at lifting that limitation to solve a number of other problems I am encountering trying to get other bugs fixed. I hadn't gotten around to filing a bug yet, but I'll do so and have it block this.

However... the media fragments issue does not explain what was happening in older versions of Firefox, since -moz-resolution is a fairly recent addition. Is there another problem lurking here?
Flags: needinfo?(seth)
Depends on: 977459
> Is there another problem lurking here?

Possibly, but the testcase described in comments 22 and 24 doesn't show it.  If I change browser.xml to not append the -moz-resolution bit, I only see one request for the image following those steps.
(In reply to Boris Zbarsky [:bz] (reviews will be slow; ask someone else) from comment #25)

> Does it really make sense to do the bug 828508 bits for cases when we're
> using the main-page image as favicon?  If not, worth filing a bug blocking
> this one on that and retesting once that's fixed...

So how is that actually implemented? If we're setting the full-size page image as the tab favicon, and just relying on it being scaled appropriately at rendering time, then there'd be no need for the -moz-resolution stuff on it AFAICS.

OTOH, if we're explicitly scaling the main page image to favicon size before setting it on the tab, then we'd need to communicate somehow whether that should be 16x16 px or 32x32.
Flags: needinfo?(jfkthame)
I have logged a seperate bug that is a very similar issue to this but the double requests are happening on an entire page not just an image. bugzilla.mozilla.org/show_bug.cgi?id=977152. Given how long this issue has been around I'm assuming it has crept into more of the product.
I now suddenly realized that installed Firefox plugins are possibly to blame for this; either Firebug (which I had enabled when I first noticed the issue) or other plugins I had (and still have) installed
(In reply to hinst from comment #31)
> I now suddenly realized that installed Firefox plugins are possibly to blame
> for this; either Firebug (which I had enabled when I first noticed the
> issue) or other plugins I had (and still have) installed

I had seen someting similar to this hinst however it turned out I was falling over a <META> tag issue that is reported https://bugzilla.mozilla.org/show_bug.cgi?id=61363 although I am still seeing quite a lot of wierd behaviour creeping into the Firefox releases.
(In reply to Jonathan Kew (:jfkthame) from comment #29)
> So how is that actually implemented? If we're setting the full-size page
> image as the tab favicon, and just relying on it being scaled appropriately
> at rendering time, then there'd be no need for the -moz-resolution stuff on
> it AFAICS.
> 
> OTOH, if we're explicitly scaling the main page image to favicon size before
> setting it on the tab, then we'd need to communicate somehow whether that
> should be 16x16 px or 32x32.

It's the former. tabbrowser.xml could probably with some adjustments avoid adding #-moz-resolution when using a standalone image as its own favicon.
Flags: needinfo?(dao)
Same. Server is registering 2 hits for each page refresh...quite frustrating, it's making the hitcounts on my pages increment too fast.
Wondering if this slows my web-browsing experience in general for all sites.
Chrome/etc does not exhibit this behavour.

Firefox Version: 28.0
OS: Windows7 64Bit

server php code...

	$fp = fopen($name, 'rb');
	header("Content-Type: image/png");
	header("Content-Length: " . filesize($name));
	fpassthru($fp);
	exit;

image info
	Type: JPG
	Size: 24.2KB
	Dimensions: 600x600
	DPI: 96 dpi
	Bit Depth: 24

access.log
	192.168.xx.xxx - - [30/Mar/2014:20:31:08 +1000] "GET /directory/image/1234 HTTP/1.1" 200 25236 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0"
	192.168.xx.xxx - - [30/Mar/2014:20:31:08 +1000] "GET /directory/image/1234 HTTP/1.1" 200 25235 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0"


.htaccess
	RewriteRule ^image/([a-z,A-Z,\_,\-,0-9,]+)?$ image.php?args=$1

To anyone working on this..Thanks ;)
Same. I've built an upload script which logs the requests.
It works fine with zip, txt, and a lot of other types, but png and jpg are buggy.
One click are two requests, so the counter adds 2 requests.
I've already tested a lot of sings f.e. disabling the cache, but nothing helped.
So I've found an interesting thing about hat bug.
If the image is bigger than 1023p, there'll be only on request.
If its smaller than 1023p, there'll be two requests.

I hope anyone have an idea, how to fix the bug.

I've tested it with the following script and two persons (latest Firefox, safemode):

<?php
$size = isset($_GET["x"]) ? $_GET["x"] : 1;

header("Content-Type: image/png");
mail('mail@example.com', 'test', 'called (' . htmlspecialchars($_GET['x']) . 'p) from '. $_SERVER['REMOTE_ADDR']);
file_put_contents("logme.txt", "SIZE: ". $size .", IP: ". $_SERVER["REMOTE_ADDR"] ."\r\n", FILE_APPEND);
readfile("http://dummyimage.com/". $size);
(In reply to Lars Nielsen from comment #36)
> So I've found an interesting thing about hat bug.
> If the image is bigger than 1023p, there'll be only on request.
> If its smaller than 1023p, there'll be two requests.
That's because Firefox won't show a preview image in the tab's favicon if any dimension of the image is 1024 pixels or higher. It's the favicon where the second request comes from.


I'd like to explain what I'm experiencing separately... I did this after restarting with addons disabled on Firefox 31.0 in Windows 7 Professional x64.

Upon browsing to an image directly, I saw three requests in Fiddler. The first two requests were identical. The third request's Accept header was different. Here's the change between the first two requests and the third request:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept: image/png,image/*;q=0.8,*/*;q=0.5

The image being downloaded has no file extension, has a query string, and is a JPEG image (which makes that header sound even more weird). When going back and forth in the history, it causes one request each time (with the first/second Accept header). Refreshing the page causes two requests (the same normal Accept header). The only difference is there's "Cache-Control: max-age=0" added on to the first request (which makes sense given the purpose of refreshing the page). Finally, after waiting a bit of time, all three requests happen upon visiting an image again (even by pressing the back button). Sometimes if it's not too long after, it's only the normal two requests (and no Cache-Control header).

As for responses, they're all almost identical. The only change per response is a single custom header changing, not part of the HTTP spec.

As for images with a dimension over 1023px, it works how things seem to be expected to work. That is, there will only be one request per visit, unless you've recently visited it. In that case, there are no requests made to download the image. So this confirms the favicon download causes that second request (and something else related to the size of the image causes the third).

I'll leave this up to you to interpret it. I'm guessing that third request with the different Accept header is related to the problem that existed before -moz-resolution was introduced.
Duplicate of this bug: 1049334
It seems to be caused by add-ons. When I use Firefox 37.0.2 (Windows 7) with all extensions and plugins disabled the image will be loaded once. When the extensions and plugins are enabled it will show up three times in the access log.
Trying with steps described in comment 22 and comment 24 I cannot reproduce the bug with Nightlies when e10s is enabled.
I experienced the same duplicated request issue on FF 40.0.3 Win7 64 bits when loading an image in Tomcat.  After a lot of fiddling based on the comments above, it seems the behavior occurs only when FF *thinks* it's serving a dynamic image, no matter the image URL actually went through any filter/servlet or not. E.g. loading a static image like below,

test.gif - generates a single request
/test.gif?a=1 - generates duplicated requests

- request/response headers look identical in both URLs, with no cache-control headers. E.g.
----------------- >< ------------------
Request: 

Host: xxx.xxx.xxx
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: ...
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

Response:

Accept-Ranges: bytes
Content-Length: 343
Content-Type: image/gif
Date: Sun, 20 Sep 2015 05:34:20 GMT
Etag: W/"343-1253553159000"
Last-Modified: Mon, 21 Sep 2009 17:12:39 GMT
Server: Apache-Coyote/1.1
----------------- >< ------------------

- size of test.gif < 1024 bytes.
- the duplicated request behavior only occurs when the image is loaded directly in a tab.  No problem when loaded within an HTML.  This seems to indicate the 2nd request came for the favicon as folks indicated above.  

This duplicated request behavior can be "fixed" once server returns a cache-control header to allow any amount of caching for the given image.  Not sure if this behavior is by design or not.
> Not sure if this behavior is by design or not.

This is by design in the sense that the HTTP implementation purposefully skips the cache in this situation.  Not only by design, but required per spec at the time the code was written.  See RFC 2616 section 13.9:

  We note one exception to this rule: since some applications have traditionally used GETs
  and HEADs with query URLs (those containing a "?" in the rel_path part) to perform
  operations with significant side effects, caches MUST NOT treat responses to such URIs as
  fresh unless the server provides an explicit expiration time.

RFC 7234 removed this "MUST NOT" requirement; see discussion in bug 718797 about what the "right" behavior should be now...
(In reply to Boris Zbarsky [:bz] from comment #42)
> > Not sure if this behavior is by design or not.
> 
> This is by design in the sense that the HTTP implementation purposefully
> skips the cache in this situation.  Not only by design, but required per
> spec at the time the code was written.  See RFC 2616 section 13.9:
> 
>   We note one exception to this rule: since some applications have
> traditionally used GETs
>   and HEADs with query URLs (those containing a "?" in the rel_path part) to
> perform
>   operations with significant side effects, caches MUST NOT treat responses
> to such URIs as
>   fresh unless the server provides an explicit expiration time.
> 
> RFC 7234 removed this "MUST NOT" requirement; see discussion in bug 718797
> about what the "right" behavior should be now...

From my point of view Firefox should cache here because its the Firefox that needs extra copies to provide extra functionality (not the user). Should Firefox re-request every time we switch tabs? Of course not.
We started getting this issue again in new version  
Seems  moving from one page to another  brigning 2 value in the backend  , while in chrome it is one  

Calls are same  in both the browser
You need to log in before you can comment on or make changes to this bug.