780197 - HTML5 offline cache does not reload modified files when manifest changes

Status: RESOLVED WONTFIX

(Core :: Networking, defect)

Description

[imam.developers]

Attachment: Screenshot of HTTP log in Fiddler2 (commented)

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:10.0.6) Gecko/20100101 Firefox/10.0.6
Build ID: 20120713125334

Steps to reproduce:

Setup: HTML5 website with offline cache functionality. The apache webserver is configured to send cache.manifest immediately expiring (ExpiresByType text/cache-manifest "access plus 0 seconds"), as recommended here: https://developer.mozilla.org/en/Offline_resources_in_Firefox/#Gotchas All other files are transmitted normally (default settings).

When initially accessing the page, the offline cache is filled correctly and subsequent access to the pages issues only a request for cache.manifest to see if updates are there.

Now let's say, the pages, pictures and cache.manifest WERE updated...


Actual results:

Firefox retrieved all files that were ADDED to the modified cache.manifest, and also removed no longer existing files from the offline cache. But the browser did not ask the server for MODIFICATIONS on still existing files, resulting in an inconsistent offline cache state: The not-updated pages still reference pictures that are not existing anymore. With updates on javascript in many files, the whole offline application gets broken.

See attached HTTP log and screenshot

Tested with Firefox 14.0.1 and Firefox ESR 10.0.6.


Expected results:

The browser should ask the server for modifications on all files that are in the updated cache.manifest and only cache atomically (all or nothing). It would be ok to use "If-Modified-Since" and expect HTTP 305, though.

Comment 1

[imam.developers]

Attachment: Fiddler2 HTTP Session Archive

Comment 2

[imam.developers]

Attachment: Web site (2 versions) used for testing

Make sure Expire .htaccess is processed correctly and mod_expires is activated in apache. First load cache-test/page1.html then "update" the web site by switching to version 2 (you can use the DOS batch file for that).

Comment 3

[nemo]

Is this what bug #781685 is referring to?

But I agree. This is incredibly annoying.

Using versioned names for all files in the manifest works, but that is really not practical.

Comment 4

[nemo]

Ignore my comment here.  My issues w/ modified files not being resolved were solved by setting:
ExpiresByType text/cache-manifest "access plus 0 seconds"

In apache.  In short, cache caching was my problem, not the problem described above.
Oh, and, for the problem above, just a guess, but maybe you need a no-cache directive like:
<Files *>                                                                                                                            
Header set Cache-Control: "private, pre-check=0, post-check=0, max-age=0"                                                            
Header set Expires: 0                                                                                                                
Header set Pragma: no-cache                                                                                                          
</Files>

On the files you are modifying?

Comment 5

[imam.developers]

This is correct, this bug can be circumvented by transmitting ALL FILES ALWAYS with "no-cache" or "expires 0 seconds". But it must be set in advance for all files; it is not enough to set the modified files to no-cache later.

However, this is not a good solution and must still be considered as a bug: It leads to inconsistent offline cache state (some updated - some not) when files are transmitted normally without expires-tags by a webserver. The browser will not look for updated files even half a year later. The browser should at least ask the server for modifications (if-modified-since) when the cache manifest was updated, very musch like the browser does when you re-visit a normal page after some days.

@nemo: The problem you describe and can be solved with 'ExpiresByType text/cache-manifest "access plus 0 seconds"' is indeed a different one (Bug 714299) for which a fix is already submitted and targeted for FF17.

Comment 6

[bryandonnovan]

I see problematic AppCache behavior in all versions of Firefox (34-38).

Manifest served with Cach-control: no-cache
All files in manifest served with cache-control: max-age=0

Firefox detects changed manifest, triggers an UPDATEREADY event on appcache object, page is reloaded, then OLD files are displayed.

Some of the time, the updates work perfectly, but then Firefox can get into a state where it just ignores the new files, even after fetching them.

Looking at about:cache, the last modified dates seem to indicate the newer files have been fetched, however, the older files are still displayed.

Today I saw one case where the page was loaded, displaying the latest content, then an UPDATEREADY was fired, then the page reloaded and then displayed OLD content!  

about:cache sometimes shows many prior versions of what is supposed to be a single set of manifest files.  I'm not sure why Firefox keeps the old versions around, but it seems clear that mistakes are made and stale content is getting served.  

about:cache will also sometimes show "no expiration time" in the expires column. How can a file with max-age=0 have no expiration?

There seem to be very serious problems with the appcache implementation serving stale content.  It would be great to see some attention paid to this problem which has been present for a while now.

I have to advise users to steer clear of Firefox because of these issues.

Comment 7

[Markus Unterwaditzer]

I can also reproduce this with Firefox 39. I need to expand the no-cache headers set for the manifest to ALL files to make the HTML5 appcache behave properly in Firefox. This isn't an issue with Chromium 44.

Comment 8

[donrhummy]

This is still occurring in Firefox 40. This is a CRITICAL bug.

Comment 9

[moshe]

I agree. this stops people from using html app cache. it works great in all browsers but FF