Closed Bug 683794 Opened 9 years ago Closed 6 years ago

onupdateready event not fired when an html5 app cache app is updated

Categories

(Core :: DOM: Core & HTML, defect)

defect
Not set
major

Tracking

()

RESOLVED DUPLICATE of bug 830588

People

(Reporter: blizzard, Assigned: mayhemer)

References

Details

We've got feedback from a large app vendor that we don't fire the "onupdateready" event isn't fired when we're done updating an html5 app cache app.

The docs say:

https://developer.mozilla.org/en/nsIDOMOfflineResourceList

onupdateready 	nsIDOMEventListener 	An event listener to be called when a resource update is ready; this event is not currently used since versioned application caches aren't supported yet.

So I guess it's known that we don't yet support that.

Sorry, I'll try to get a small test case to verify.
Assignee: nobody → Olli.Pettay
This is covered by tests and also works for me in an easy test case: 


<html manifest="manifest.cacheManifest">
  <head>
  <script>
    applicationCache.onupdateready = function() {
      alert("onupdateready");
    }

And the manifest has been changed.  They should first check the manifest has correct Cache-control header.  Then, if they get e.g. "checking" event.

IMO this works, but something makes it hard to use correctly and web devs make mistakes because of that.
So is this bug then invalid? Or do we fire updateready at different times than other browsers?
(In reply to Olli Pettay [:smaug] from comment #3)
> So is this bug then invalid? Or do we fire updateready at different times
> than other browsers?

AFAIK, we fire it not sooner then after onload.  It also happens only after user accepts the offline permission via the pop-up that appears when we detect the manifest= attribute and the host has not been "offline-enabled" yet.

I think they really have some problem, but we need more information from them, how the code looks, any errors in the console, etc.  For me definitely this event works, also checked in my semi-production offline web app I use on daily base.
blizzard, I think we need your help here. Could you ask your contacts what is not working in FF?
I sent mail today.  Sorry for the delay.
It's definitely not working properly. I have no problems with Chrome and Safari while in FF events are not fired and cache is not updated when manifest is changed:

I've build this test case:
http://offline.medyk.org/

It's very plain, when I update cache manifest and reload page safari and chrome instantly get update, fire events and after another reload I have updated application, on FF each time I load page I have just 'checking' and 'noupdate' events

If you want to reproduce it, serve with proper headers following files:
http://offline.medyk.org/index.html
http://offline.medyk.org/offline.appcache
and try to test it updating offline.appcache file
Per spec I don't see us doing anything wrong. The manifest can be cached.

But this is apparently unclear to authors, so perhaps the spec needs to be changed.
Olli, but if you cache manifest, how can we update application files ?
We need the way to update application, and it looks in just FF it's impossible, it makes whole offline feature hardly usable.

Also 'update()' method ( http://www.whatwg.org/specs/web-apps/current-work/#dom-appcache-update ) doesn't help, it's supposed to invoke application cache download process but it isn't
Mariusz, send Cache-control: no-cache header with the manifest response or setup your server to do that automatically.  That is enough.
Honza,
Thanks that helped, but still only partially. 
With 'no-cache' FF indeed checks manifest file for changes and fires 'updateready' event when it's changed, but still, resources remain not updated.
Cause it's the same, FF doesn't check server for new version of resources but gets them from cache instead.
I finally made it working when I added "no-cache" header for all resources, I think it's a bit overkill and shouldn't be needed.
http://appcachefacts.info/ (I know it's not a spec) suggests that browsers after manifest update, check for new version of resources with "If-Modified-Since" header, I believe it's sane solution, and it should work that way.

Other thing which is probably more related to this bug is that 'updateready' listener is fired only when it's registered during page load. If I register listener e.g. in window.onload (but still before updateready is expected) it doesn't work

e.g.

window.onload = function () {
  window.applicationCache.addEventListener('updateready', function () {
    // not fired
  }, false);
};

There's simple but bit strange workaround for that, we need to register dummy listener during page load, and all following listeners will be fired as well:

window.applicationCache.addEventListener('updateready', function () {
 // dummy, fired
}, false);
window.onload = function () {
  window.applicationCache.addEventListener('updateready', function () {
    // right one, fired as well !
  }, false);
};
(In reply to Mariusz Nowak from comment #11)
> suggests that browsers
> after manifest update, check for new version of resources with
> "If-Modified-Since" header, I believe it's sane solution, and it should work
> that way.

Then you have to add the Last-modified header to do this.

To explain: If-Modified-*, If-None-Match-* headers are conditional request headers that can be sent only when server sent E-Tag or Last-Modified header with the response.  W/o that info from the server a conditional request could not be made.

Read more e.g. here: http://blogs.msdn.com/b/ieinternals/archive/2010/07/08/technical-information-about-conditional-http-requests-and-the-refresh-button.aspx or just google "conditional HTTP request".

Actually, the server or just the app directory should be setup to assign ETags to all resources automatically.  I'm no expert to setup nginx server for this, but IIS has an option to check for the files expiration automatically.  Many Apache deployments have the same settings on hosting services.

> 
> Other thing which is probably more related to this bug is that 'updateready'
> listener is fired only when it's registered during page load. 

It means in a script in the <head> tag?

> If I register
> listener e.g. in window.onload (but still before updateready is expected) it
> doesn't work

We have tests that check this works, anyway, that is a different bug.

> 
> e.g.
> 
> window.onload = function () {
>   window.applicationCache.addEventListener('updateready', function () {
>     // not fired
>   }, false);
> };
> 
> There's simple but bit strange workaround for that, we need to register
> dummy listener during page load, and all following listeners will be fired
> as well:
> 
> window.applicationCache.addEventListener('updateready', function () {
>  // dummy, fired
> }, false);
> window.onload = function () {
>   window.applicationCache.addEventListener('updateready', function () {
>     // right one, fired as well !
>   }, false);
> };

Can you please file this as a new bug?
Mariusz, any updates?  Should we close this for now as INVALID until the spec change gets decided?
Honza,

On my side Firefox doesn't send "If-Modified-Since" headers, although server sends "Last-modified",
I wanted to investigate it further and  file respective bugs (if it's indeed a bug) as you asked, but I'm currently finishing one project, and I'm just running out of time, I'll try to figure this out by end of next week.
So the spec is clarified that caching is the right thing to do
http://www.w3.org/Bugs/Public/show_bug.cgi?id=14431
"HTTP caching semantics should be honored for this request."

But we need to make sure If-Modified-Since etc work correctly.
Mariusz, could you perhaps log the http headers Firefox is sending to your server and
and Firefox gets back and attach the log to this bug as an attachment.
Olli,

I looked into it again, and this time I found that 'if-modified-since' header is send by Firefox, I'm not sure now why I didn't get those before. If I'll approach this problem again I'll investigate it further, but for now we may assume it's working as expected. Thanks!
Mariusz, just to verify, with the right http headers you do get updateready event working properly?
Olli,

There's still issue I mentioned above, that listener needs to be registered during page load, otherwise browser won't fire event at all. Apart of that it works, even in case 'updateready' is not fired after reload we have new version loaded.
(In reply to Mariusz Nowak from comment #19)
> Olli,
> 
> There's still issue I mentioned above, that listener needs to be registered
> during page load, otherwise browser won't fire event at all.
New bug should be filed for this.

> even in case 'updateready' is not fired after reload we have new
> version loaded.
I don't understand this.
> > even in case 'updateready' is not fired after reload we have new
> > version loaded.
> I don't understand this.

"updateready" is not fired, but after I reload page I see updated version - this fact narrows issue, so it's only about events not being fired, new version is downloaded as expected
So, are you saying that (a) you don't get updateready event at all, ever?
Or that (b) you do get updateready, but only if it is registered during load event handling? 
Or that (c) you do get updateready, but only if it is registered before or during load event handling?
Or that (d) you do get updateready, but only if it is registered before load event handling?
Or that (e) you do get updateready, but only if it is registered after load event handling?

Thanks for testing!
If I read the code correctly, registering an updateready event listener shouldn't change
behavior at all.
But, I wonder if updateready event can be fired in some cases before load event.
updateready is always fired after load event on my side.
However I need to registrer (any) listener during page load (I haven't nailed down which state change makes that difference, I assume that it should be before load event is fired on window.

It looks that firefox internally turns on applicationCache event handling feature only if some listener was registered during page load, if it wasn't then firefox won't fire any applicationCache events - that's how I see it.
Are you sure it is about event handling? Is it enough to just access .applicationCache before
load event to activate it somehow?
I have been following the thread here. What happens in the case where the webserver does not fire a response with the last-modified and e-tag attached? It seems like firefox still caches the cache.manifest and since we cannot rely on that to figure out whether there is a new cache available, theres the potential for users to be stuck on that cache.manifest version and unable to update.

As well, in my testing, it seems like if you provide the "last-modified", it does the appropriate sending out of "If-Modified-Since" headers. But, I get in the state in the appcache upgrade scenario that it pulls down the cache.manifest and all the static data but does not seem to do the swap to the new manifest and files it downloads.
(In reply to Charlton from comment #26)
> but does not seem to do the swap to
> the new manifest and files it downloads.

You have to reload the page.
Right. It does not use the later cache.manifest after reload. Which is the problem.
(In reply to Charlton from comment #28)
> Right. It does not use the later cache.manifest after reload. Which is the
> problem.

This works in tests and my own offline web app.  If you can, please file a new bug and CC me and add a simplified test case.  What you claim is a different issue then this one.
Ignore my comment, it does seem to work. Seems be a different issue where the "View page source" does not seem to show the updated html source, but an old cached source. I can try to get a small test case of this if you guys are interested.

I am going to rely on firebug from now on since it seems more reliable.
This is completely broken as far as I can tell.

Here's My test code:

CACHE MANIFEST
# Updated 14 July 2014 10:35

# Explicitly cached entries
index.html

# Additional resources to cache
CACHE:
/// various cached files


cachechecker.html:

(Snip html)

// Convenience array of status values
var cacheStatusValues = [];
 cacheStatusValues[0] = 'uncached';
 cacheStatusValues[1] = 'idle';
 cacheStatusValues[2] = 'checking';
 cacheStatusValues[3] = 'downloading';
 cacheStatusValues[4] = 'updateready';
 cacheStatusValues[5] = 'obsolete';

 // Listeners for all possible events
 var cache = window.applicationCache;
 cache.addEventListener('cached', logEvent, false);
 cache.addEventListener('checking', logEvent, false);
 cache.addEventListener('downloading', logEvent, false);
 cache.addEventListener('error', logEvent, false);
 cache.addEventListener('noupdate', logEvent, false);
 cache.addEventListener('obsolete', logEvent, false);
 cache.addEventListener('progress', logEvent, false);
 cache.addEventListener('updateready', logEvent, false);

 // Log every event to the console
 function logEvent(e) {
     var online, status, type, message;
     online = (isOnline()) ? 'yes' : 'no';
     status = cacheStatusValues[cache.status];
     type = e.type;
     message = 'online: ' + online;
     message+= ', event: ' + type;
     message+= ', status: ' + status;
     if (type == 'error' && navigator.onLine) {
         message+= ' There was an unknown error, check your Cache Manifest.';
     }
     log(' ' + message );
 }

 function log(s) {
    //alert(s);
    //alert(document.getElementById('messages').innerHTML);
	document.getElementById('messages').innerHTML = document.getElementById('messages').innerHTML + '<li>' + s + '</li>';
 }

 function isOnline() {
     return navigator.onLine;
 }

(snip html)


Headers sent by server for cache.manifest:

Date: Mon, 14 Jul 2014 09:44:24 GMT

Server: Apache/2.2.26 (Unix) PHP/5.3.28 mod_wsgi/3.3 Python/2.7.2 mod_ssl/2.2.26 OpenSSL/0.9.8y DAV/2

Last-Modified: Mon, 14 Jul 2014 09:44:05 GMT

Etag: "2c5d058-c49-4fe2421baa340"

Accept-Ranges: bytes

Content-Length: 3145

MS-Author-Via: DAV

Keep-Alive: timeout=15, max=100

Connection: Keep-Alive

Content-Type: text/cache-manifest


200 OK


Changing the date in the cache manifest does nothing. The updateready event NEVER fires. This code all works fine in Chrome, but the only way I can refresh the cache in Firefox is to manually delete the cache contents using the Firefox history menu item and then refresh the browser. At the moment, we are advising end users to not use Firefox when accessing our app, as we have no means to push updates to their devices.
Curiously, the cache.manifest content never changes if viewed directly in FF, even though a header check tells me that the last-modified date header sent by the server HAS changed. Firefox just merrily continues to cache the old version of the cache.manifest.

Can I suggest that the simplest fix for this might be for FF to NEVER cache the manifest. after all, that is rather the point of having a manifest... that it should be continually updated. To actually cache the manifest does seem to be a bit perverse, to say the least.
Sorry. Last comment...

I have just gone into FF preferences and removed my domain from the list of permitted domains in the "Offline Web Content and User Data" allowed domains list. I then refreshed the app, and now the cache refreshes are working, with the updateready event firing properly.

It would appear that this was transient bug in the Offline Web Content data. I don't think I'm the only person who has run into this. Forcing FF to completely refresh it profile for my offline data storage seems to have fixed this for me.
(In reply to Matt Bradley from comment #33)
> Sorry. Last comment...
> 
> I have just gone into FF preferences and removed my domain from the list of
> permitted domains in the "Offline Web Content and User Data" allowed domains
> list. I then refreshed the app, and now the cache refreshes are working,
> with the updateready event firing properly.

Fantastic, so now we don't have any way to find out what was wrong...
Sorry Honza. Should it occur again I shall report back
OK. Steps to replicate this:

[1] Create an apache server which ***DOES NOT*** have this line in the config"ExpiresByType text/cache-manifest "access plus 0 seconds""
[2] Create a cache.manifest and accompanying html document
[3] Visit the html document.
[4] Change the cache.manifest
[5] It will not be updated in firefox, no matter what you do, unless you clear history.
[6] Add in the ExpiresByType text/cache-manifest "access plus 0 seconds" to your Apache server
[7] cache.manifest will still not update in Firefox
[8] Clear history. 
[9] cache.manifest will update
[10] change cache.manifest
[11] It will not be updated in firefox, no matter what you do, unless you clear history.
[12] Go to settings, and remove the domain from "Offline Web Content and User Settings"
[13] cache.manifest will now start working properly.

From what I can tell, FF is permanently caching the first expiry date it gets for the cache.manifest, even if the server later sends a different expiry date.
Appears to be worse than I thought. this is actually utterly broken.

Consistently able to replicate this issue here.
http://app.galtresfestival.org.uk/

Server is sending "correct" expires headers, yet once loaded, Firefox flatly refuses to update the cache.manifest. It never asks for it again, even though it has been told that it expires instantly.

This works fine in Chrome and other WebKit UAs
http://app.galtresfestival.org.uk/cache.manifest

GET /cache.manifest HTTP/1.1
Host: app.galtresfestival.org.uk
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:31.0) Gecko/20100101 Firefox/31.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://app.galtresfestival.org.uk/
X-Moz: offline-resource
Cookie: _ga=GA1.3.1409892816.1404296180
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

HTTP/1.1 200 OK
Date: Thu, 07 Aug 2014 16:33:29 GMT
Server: Apache/2.2.22 (Ubuntu)
Last-Modified: Thu, 07 Aug 2014 16:00:02 GMT
Etag: "29a45f-31d-5000c2e6c2480"
Accept-Ranges: bytes
Content-Length: 797
Cache-Control: max-age=0
Expires: Thu, 07 Aug 2014 16:33:29 GMT
Keep-Alive: timeout=5, max=86
Connection: Keep-Alive
Content-Type: text/cache-manifest


Firefox 31.0 fetches this document once and once only. It never asks for it again until I actually clear the browser's cache. There is simply no way to push cache.manifest updates to FF.
We are also seeing this with Firefox 31 - it's a disaster, customers are complaining and losing business because our app isn't working. Here is an incomplete work-around that will get our application updating again, IF each user clears their cache at least once!

Step 1: If the manifest file has Cache-Control: max-age=0, it will never refresh. If it is changed to no-cache, the manifest will be refreshed but the browser doesn't seem to fetch the files listed in the manifest file.

Step 2: All the files served after a manual cache clear must also have no-cache instead of max-age=0.

Step 3: updateready event doesn't fire. applicationCache.status is 1 (IDLE), even after downloading all the udpated files. The final step in this hack is to set a timer and do a window.location.reload() after verifying that the version of the code running on the client does not match what the server api is telling us the latest version is. Ouch!

We really hope this bug gets fixed and pushed soon!
Component: DOM: Events → DOM
Honza, could you take another look at this. (This certainly isn't a DOM Events issue)
Assignee: bugs → honzab.moz
Kimon, does the server send the right headers?
See comment 12 and others.
(In reply to Olli Pettay [:smaug] from comment #41)
> Kimon, does the server send the right headers?
> See comment 12 and others.

Here are the headers as reported by Firefox (about:cache) when the manifest file is served with max-age=0 and it will no longer refresh:

key: 	http://localhost:8080/manifest.appcache
fetch count: 	2
last fetched: 	2014-08-19 14:31:42
last modified: 	2014-08-19 14:30:58
expires: 	1969-12-31 16:00:00
Data size: 	245
file on disk: 	none
Security: 	This document does not have any security info associated with it.
Client: 	HTTP
request-method: 	GET
request-Accept-Encoding: 	gzip, deflate
response-head: 	HTTP/1.1 200 OK Vary: Accept-Encoding Content-Type: text/cache-manifest Cache-Control: public, max-age=0 Content-Encoding: gzip Date: Tue, 19 Aug 2014 21:30:58 GMT


The same non-working manifest file, but with request/response headers as seen in Chrome:

Remote Address:127.0.0.1:8080
Request URL:http://localhost:8080/manifest.appcache
Request Method:GET
Status Code:200 OK

Request Headers
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8,el;q=0.6
Cache-Control:max-age=0
Connection:keep-alive
Cookie:uvf=1; __uvt=; flnx=<deleted-data>
Host:localhost:8080
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36

Response Headers
Cache-Control:public, max-age=0
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/cache-manifest
Date:Tue, 19 Aug 2014 21:35:00 GMT
Set-Cookie:flnx=<deleted-data> path=/; expires=Wed, 27 Aug 2014 12:11:45 GMT; httpOnly
Transfer-Encoding:chunked
Vary:Accept-Encoding


*** And here are the headers for the manifest file served with no-cache so that Firefox does check it again:

Remote Address:127.0.0.1:8080
Request URL:http://localhost:8080/manifest.appcache
Request Method:GET
Status Code:200 OK

Request Headers
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8,el;q=0.6
Connection:keep-alive
Cookie:uvf=1; __uvt=; flnx=<deleted-data>
Host:localhost:8080
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36

Response Headers
Cache-Control:no-cache
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/cache-manifest
Date:Tue, 19 Aug 2014 21:41:40 GMT
Set-Cookie:flnx=<deleted-data>; path=/; expires=Wed, 27 Aug 2014 12:18:41 GMT; httpOnly
Transfer-Encoding:chunked
Vary:Accept-Encoding

If I only change the manifest file from max-age=0 to no-cache, then the manifest file does get updated in Firefox, however, the files listed in the manifest file are never requested until I clear the app cache. If I change those files to no-cache as well, then they are fetched each time the manifest file is changed, however, the updateready event is never fired. window.location.reload() does restart our web app with the updated code base.

Let me know if you need any more info, thanks, Kimon
Public test case would be great.  You could also provide the files+config so I can host them locally as you do.
(Comment 43)
Flags: needinfo?(kimon)
Honza, will email you with additional email later today/tonight.
Flags: needinfo?(kimon)
Honza, I just setup a server with our code and emailed you with additional info.
(In reply to Kimon from comment #46)
> Honza, I just setup a server with our code and emailed you with additional
> info.

Thanks, email received.  Going to look at it yet this week.
So, according your mail, it seems the missing update was missing Cache-control: no-cache header.  That has been solved.

I'm closing this bug as dup of bug 830588.

I'll open a new bug for the POST being loaded from appcache problem.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 830588
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.