400615 - image caching uses incorrect expiry time

Status: RESOLVED WORKSFORME

(Firefox :: General, defect)

Description

[Jon Burgess]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9a9pre) Gecko/2007102004 Minefield/3.0a9pre
Build Identifier: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9a9pre) Gecko/2007102004 Minefield/3.0a9pre

The www.openstreetmap.org site serves map image tiles from the tile.openstreetmap.org server. These tiles are updated dynamically and if you browse a tile which is old then the backend code returns a tile with a 30 minutes expiry time which gives the servers some time to re-render the tile.

It seems that FF is not caching these tiles with 30 minute expiry. 

Reproducible: Always

Steps to Reproduce:
1. Browse map at http://www.openstreetmap.org in Minefield
2. Zoom in to street level in some area of the map
3. View page info and look through images in media. 
Actual Results:  
Some of the tile images are listed as 'not cached'.

Expected Results:  
Every tile should get cached for at least 30 minutes.

It appears that maybe the browser is confused because of DST which means that the declared expiry time is < clock time in UK BST. An example tile retrieved from the server:-

$ wget -O /dev/null -S http://b.tile.openstreetmap.org/15/16218/10769.png
--19:45:12--  http://b.tile.openstreetmap.org/15/16218/10769.png
           => `/dev/null'
Resolving b.tile.openstreetmap.org... 128.40.58.201
Connecting to b.tile.openstreetmap.org|128.40.58.201|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Sun, 21 Oct 2007 18:45:12 GMT
  Server: Apache/2.0.55 (Ubuntu) mod_ruby/1.2.5 Ruby/1.8.4(2005-12-24)
  Expires: Sun, 21 Oct 2007 19:15:12 GMT
  Connection: close
  Content-Type: image/png
Length: unspecified [image/png]

[ Note: the specific URL above may well return other expiry times since it will get rendered at some point and then will get a 3 day expiry time set. ]

Above the local wall time is 19:45 BST (Europe/London which is currently UTC+01:00 due to daylight saving) The server declares the tile to be served at 18:45 GMT with expiry at 19:15 GMT. The browsers seems to treat this as uncacheable.

If I tweak the backend server code to pass back +90 minutes as the expiry then it gets cached OK. It looks like the browser is ignoring the timezone in the response date/time. Alternatively maybe the browser thinks GMT is the same as BST when daylight saving is in effect, but this is not true, GMT is effectively the same as UTC even during daylight saving.

Comment 1

[Jo Hermans]

When I load the url from above (using latest Minefield), I see this in the cache :

           Key: http://b.tile.openstreetmap.org/15/16218/10769.png
     Data size: 262144 bytes
   Fetch count: 3
 Last modified: 2007-10-21 22:34:28
       Expires: 2007-10-21 23:04:25

This was done at 22:34:28 CET (= GMT+2), the time in about:cache is always displayed in your local timezone. That's a 30 minute timeout.

I'll report back in 30 minutes.

Comment 2

[Jo Hermans]

at 23:06 it was :
           Key: http://b.tile.openstreetmap.org/15/16218/10769.png
     Data size: 262144 bytes
   Fetch count: 2
 Last modified: 2007-10-21 23:06:26
       Expires: 2007-10-21 23:36:24

That's a new file, as you can see in the expires and in the fetch-count (which higher than 3 when I used to check for caching), I forgot to mention that).

When I try to load the same url (clicking on the link), I see that the same file from the cache is always used, because only the fetch-count goes up.

When I reload the image by a click on the reload button, the url is loaded again from the webserver.

           Key: http://b.tile.openstreetmap.org/15/16218/10769.png
     Data size: 262144 bytes
   Fetch count: 3
 Last modified: 2007-10-21 23:11:53
       Expires: 2007-10-21 23:41:51

Comment 3

[Jo Hermans]

Note that when I look at a single image at the test above, Page Info always reported that the image came from the disk cache. When I browse around on openstreetmap.org, I see a mixture of cached and not cached images, but I don't see why some are cached, and some are not.

Comment 4

[Jon Burgess]

I didn't know of about:cache before. Trying a few different tiles I seem to get inconsistent results. The tile generally appears in the memory cache with the correct expiry. Often it doesn't appear in the disk cache, but will if I reload it a couple of times. 

Is it the case that image->properties will report it as 'not cached' unless it hits the disk cache? 

Perhaps Minefield is just being quite picky about what it pushes to the disk cache and short lived objects don't often get cached?

At higher zooms many of the tiles on the map will need rendering (the number of map tiles on the site is growing exponentially and the rendering is struggling to keep up).

I'm happy for this to be 'invalid' if Minefield is just being choosy about not caching objects with a short expiry unless they get accessed several times. It is just a shame to have to wait for the tiles to get reloaded as the user move arounf the map.

Comment 5

[Jo Hermans]

I'm still not sure why some images are in the disk cache and some are not, but it might not be such a big deal.

The ones that are marked as "not cached" are indeed not in the disk cache for some reason, but I can still see them in the memory cache ("not cached" is a misleading label). That is, they're cached, although they have a higher probability that they will be kicked out of memory (and lost) to make room for new images. But the default memory cache is 18MB (when you have 1GB of RAM), so that won't happen to often. The memory cache will still help you to scroll around and zoom in and out a bit, even when some of the images won't get into the disk cache.

Comment 6

[Tyler Downer [He/Him]]

Does this still happen in Firefox 3.6.6 or later in a fresh profile.

Comment 7

[Jon Burgess]

It looks like this has been fixed. I tested with Firefox 3.6.4 and it looks like all the map tiles are showing up as being cached even though the expiry time on some of them is as low as 15 minutes. 

If I stop the web server from running then I can successfully pan around and view the tiles which had been browsed previously without any obvious holes appearing due to uncached tiles.

Comment 8

[Tyler Downer [He/Him]]

WFM then.