Closed Bug 834338 Opened 12 years ago Closed 12 years ago

appcache should not apply its expires heuristics on the downloaded files

Categories

(Core Graveyard :: DOM: Apps, defect)

Product:
Core Graveyard
Component:
DOM: Apps
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 830588

People

(Reporter: julienw, Unassigned)

Details

STR: * install a hosted+appcache app * change it (appcache manifest + some things in your index.html so that it's obvious that it isn't updated) * trigger an update * update the app * wait for the end of the update * launch the app Expected: * the new version of the app is displayed Actual: * the old version of the app is displayed Note that I used an app with only the appcache_path in the webapp manifest, I didn't use <html manifest=''> at all.
On my server logs I see no request other than the webapp manifest and appcache.
Well, if you don't specify <html manifest=...>, gecko will not use the appcache for your page.
So you mean that "appcache_path" has no effect for updates if an app doesn't use <html manifest=...> ?
No, I'm not saying that. Just that your site once launched will not use appcache. But you said : "On my server logs I see no request other than the webapp manifest and appcache." So you got appcache request from the update pings, right?
Now I see these pings yes (that was bug 834285, and I still don't know what changed between before and now). But I don't see the app updating if the appcache changes.
I don't really understand why this doesn't work, but given that there's a workaround we won't block on this. What is very confusing to me is that how do we know where to load the start page from (appcache vs. network) based on the contents of that very same page? I.e. do we start loading the page from the appcache and if it doesn't contain the manifest attribute we reload it from the network? Or do we somehow magically know that the file in the appcache doesn't contain the manifest attribute and so we go directly to the network?
blocking-b2g: tef? → -
Keywords: dev-doc-needed
:sicking: I think that we always go to the offline cache, as if it was specified in the html tag. I disagree respectfully with this unnomination, this is a very serious bug in my opinion, and that is part of v1 features.
Sounds like the way I read this bug that having this bug basically almost destroys the entire point of the appcache_path for an update, right?
Right.
Then I pretty much flat out disagree with sicking on this one. It's a broken feature.
blocking-b2g: - → tef?
Keywords: dev-doc-needed
We also can't make the guarantee that the above work-around suggested won't impact preloads that are non-removable - I don't want to spin those app developers on a wrecking gamble trip that will impact our users. I'd rather avoid playing with the trap of getting into a bad state here. App developers for preloads are already well on their way down the path to finishing off these apps (some of which are actually done, most notably the notes app that actually would be impacted by this bug). And I disagree this is a worthwhile evangelism fight for something that is pretty much in our ball court for being incorrect.
Actually change that - notes wouldn't be impacted by this bug. They actually did call out the manifest attribute as well - http://everythingme.github.com/ffos-notes/. Although I'd need to dig into what preloads are doing what.
Spoke with sicking on this IRC. I want to investigate to see if you can't screw yourself with a preinstalled hosted app preloading appcache. Putting qawanted to investigate. Still think this is worth tracking, however.
Blocks: Apps-Dev-Doc-Needed
blocking-b2g: tef? → ---
tracking-b2g18: --- → ?
Keywords: dev-doc-needed, qawanted
QA Contact: jsmith
Got my clarification in IRC - no need to do qawanted to investigate then.
Keywords: qawanted
Summary: My hosted+appcache app won't update → A hosted app preloading appcache through appcache_path without a appcache manifest specified in html manifest won't update to use new appcache resources
To put you on the right path: (In reply to Jonas Sicking (:sicking) from comment #6) > I don't really understand why this doesn't work, but given that there's a > workaround we won't block on this. > > What is very confusing to me is that how do we know where to load the start > page from (appcache vs. network) based on the contents of that very same > page? Top level document loads first look into the app cache. If found, the whole load group is associated with that app cache and top doc and subresources load using that app cache. presence of the manifest attribute is not needed and doesn't have any affect. > > I.e. do we start loading the page from the appcache and if it doesn't > contain the manifest attribute we reload it from the network? No, as explained in the previous answer. > > Or do we somehow magically know that the file in the appcache doesn't > contain the manifest attribute and so we go directly to the network? No magic ;) (In reply to Julien Wajsberg [:julienw] from comment #8) > :sicking: I think that we always go to the offline cache, as if it was > specified in the html tag. It's not that simple. We do new top level doc loads from app cache when top level doc is found in any app cache (we select the latest fetched one, the newest, AFAIK) and the item (the doc) is not marked as "foreign". (foreign is set when app cache we've loaded a doc from has different manifest then optionally specified manifest attribute in the <html> tag of that top doc (discovered during parsing of the doc)). This all is by the spec. The spec also says to run the cache update process when manifest attribute in the document is found (+some other minor conditions not related to this bug). That's what we do. However, the spec also says: "User agents may invoke the application cache download process, in the background, for any application cache group, at any time (with no cache host). This allows user agents to keep caches primed and to update caches even before the user visits a site." It means we can run the update any time we want. > > I disagree respectfully with this unnomination, this is a very serious bug > in my opinion, and that is part of v1 features. We may add a code to run the update also when there is no manifest but a top doc has loaded. It may be what to do to solve this bug.
(In reply to Honza Bambas (:mayhemer) from comment #16) > (In reply to Julien Wajsberg [:julienw] from comment #8) > > :sicking: I think that we always go to the offline cache, as if it was > > specified in the html tag. > > It's not that simple. We do new top level doc loads from app cache when top > level doc is found in any app cache (we select the latest fetched one, the > newest, AFAIK) Is that the reason that the appcache manifest is pinged when we launch an app (this app does _not_ have the appcache in the manifest URL) ? > and the item (the doc) is not marked as "foreign". (foreign > is set when app cache we've loaded a doc from has different manifest then > optionally specified manifest attribute in the <html> tag of that top doc > (discovered during parsing of the doc)). This all is by the spec. Which one is foreign ? The one in the html tag, or the one in the webapp manifest ? > The spec also says to run the cache update process when manifest attribute > in the document is found (+some other minor conditions not related to this > bug). That's what we do. However, the spec also says: > > "User agents may invoke the application cache download process, in the > background, for any application cache group, at any time (with no cache > host). This allows user agents to keep caches primed and to update caches > even before the user visits a site." > > It means we can run the update any time we want. If no appcache_path is set in the webapp manifest, but there is an appcache url in the html tag's manifest attribute, should we sotre this appcache manifest url as if it was defined in the webapp manifest ? That's what I understand from your sentence, and I don't know if we're doing that. I'd like that though, this would be a lot web-ish :-) > > > > I disagree respectfully with this unnomination, this is a very serious bug > > in my opinion, and that is part of v1 features. > > We may add a code to run the update also when there is no manifest but a top > doc has loaded. It may be what to do to solve this bug. Here, the bug is that we don't download the files specified in the manifest at all. But I do see the pings for the manifest. I wonder if the problem does not come from HTTP cache headers. I'll try a app with all files with an expires of "access".
(In reply to Julien Wajsberg [:julienw] from comment #17) > > If no appcache_path is set in the webapp manifest, but there is an appcache > url in the html tag's manifest attribute, should we sotre this appcache > manifest url as if it was defined in the webapp manifest ? That's what I > understand from your sentence, and I don't know if we're doing that. > > I'd like that though, this would be a lot web-ish :-) I'm 100% sure we don't do that.
(In reply to Julien Wajsberg [:julienw] from comment #17) > > I wonder if the problem does not come from HTTP cache headers. I'll try a > app with all files with an expires of "access". Ok, I just tried with a webapp with a .htaccess : ExpiresActive On ExpiresDefault "access" And I'm happy to report that... it works as expected ! So I believe that Gecko uses its usual heuristics when there are no HTTP cache headers, and the files are served directly out of cache. I don't know if that's correct according to the spec though. I'd say it's not against the spec, but is this actually specified ? Anyway that makes this bug non-blocking for me.
So what does this bug morph to then? In what scenario are you hit by this since now I'm hearing it's a narrow case based on comment 19.
Let's try with this, and maybe Honza will tell this is a wontfix.
Flags: needinfo?(honzab.moz)
Summary: A hosted app preloading appcache through appcache_path without a appcache manifest specified in html manifest won't update to use new appcache resources → appcache should not apply its expires heuristics on the downloaded files
(In reply to Julien Wajsberg [:julienw] from comment #19) > (In reply to Julien Wajsberg [:julienw] from comment #17) > > > > I wonder if the problem does not come from HTTP cache headers. I'll try a > > app with all files with an expires of "access". > > Ok, I just tried with a webapp with a .htaccess : > > ExpiresActive On > ExpiresDefault "access" > > And I'm happy to report that... it works as expected ! > > So I believe that Gecko uses its usual heuristics when there are no HTTP > cache headers, and the files are served directly out of cache. I don't know > if that's correct according to the spec though. I'd say it's not against the > spec, but is this actually specified ? > > Anyway that makes this bug non-blocking for me. How is that you have seen appcache manifest requests coming to the server? What was the communication look like? Do you have header logs?
Flags: needinfo?(honzab.moz)
Unless it is mistake in comment 1 that appcache manifest request have been coming to the server, this is obviously dupe of bug 830588.
Flags: needinfo?(felash)
In this bug I saw the appcache manifest coming to the server. In bug 834285 Ididn't, so that bug is probably a dupe, yes. In this bug, I see the appcache manifest request but then I don't see any request for the files. I have only access logs now but I can log the complete requests on monday if you need it. Here are the logs : I check for update: 92.140.64.208 - - [26/Jan/2013:00:26:46 +0100] "GET /mozilla/selfupdatinghosted/manifest.webapp HTTP/1.1" 304 177 "-" "Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0" 92.140.64.208 - - [26/Jan/2013:00:26:46 +0100] "GET /mozilla/selfupdatinghosted/manifest.appcache HTTP/1.1" 200 369 "-" "Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0" I start the update download: 92.140.64.208 - - [26/Jan/2013:00:26:50 +0100] "GET /mozilla/selfupdatinghosted/manifest.appcache HTTP/1.1" 200 369 "http://www.everlong.org/mozilla/selfupdatinghosted/index.html" "Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0" 92.140.64.208 - - [26/Jan/2013:00:26:54 +0100] "GET /mozilla/selfupdatinghosted/manifest.appcache HTTP/1.1" 200 369 "http://www.everlong.org/mozilla/selfupdatinghosted/index.html" "Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0" And then I get the downloadapplied event. Tell me if you need more.
Flags: needinfo?(felash)
Julien, what is written in the Error Console when you change the appcache manifest and then try the update? (I forgot to ask..)
Flags: needinfo?(felash)
I click on check for update : I/Gecko ( 110): -*-*- Webapps.jsm : checkForUpdate for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp I/Gecko ( 110): -*-*- Webapps.jsm : adding manifest etag:"4d40c404ad656" I/Gecko ( 110): -*-*- Webapps.jsm : Checking manifest at http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp I/Gecko ( 110): -*-*- Webapps.jsm : Got http status=304 for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp I/Gecko ( 110): -*-*- Webapps.jsm : updateHostedApp http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp I/Gecko ( 110): -*-*- Webapps.jsm : Saving /data/local/webapps/webapps.json I/Gecko ( 110): -*-*- Webapps.jsm : updateHostedApp: updateSvc.checkForUpdate for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.appcache I/Gecko ( 110): -*-*- Webapps.jsm : updateHostedApp: updateSvc.checkForUpdate return for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp - event is offline-cache-update-available I/Gecko ( 110): -*-*- Webapps.jsm : Saving /data/local/webapps/webapps.json E/GeckoConsole( 1869): Content JS LOG at http://www.everlong.org/mozilla/selfupdatinghosted/js/install.js:30 in handleDownloadAvailable: download available Then I download : I/Gecko ( 110): -*-*- Webapps.jsm : startDownload for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp I/Gecko ( 110): -*-*- Webapps.jsm : appcache found I/Gecko ( 110): -*-*- Webapps.jsm : Creating AppcacheObserver for http://www.everlong.org - updating I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 5 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to updating for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 6 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to updating for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 6 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to updating for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 6 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to updating for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 6 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to updating for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to updating for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 7 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 7 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 7 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to updating for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to updating for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 8 I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 7 E/GeckoConsole( 110): Offline cache update done, URL=http://www.everlong.org/mozilla/selfupdatinghosted/manifest.appcache I/Gecko ( 110): -*-*- Webapps.jsm : Offline cache state change for http://www.everlong.org : 10 I/Gecko ( 110): -*-*- Webapps.jsm : Offlinecache setStatus to installed for http://www.everlong.org I/Gecko ( 110): -*-*- Webapps.jsm : Saving /data/local/webapps/webapps.json E/GeckoConsole( 1869): Content JS LOG at http://www.everlong.org/mozilla/selfupdatinghosted/js/install.js:56 in handleDownloadSuccess: download success E/GeckoConsole( 1869): Content JS LOG at http://www.everlong.org/mozilla/selfupdatinghosted/js/install.js:62 in handleDownloadApplied: download applied E/GeckoConsole( 1869): Content JS LOG at http://www.everlong.org/mozilla/selfupdatinghosted/js/install.js:65 in handleDownloadApplied: download was applied
Flags: needinfo?(felash)
BTW, when I change the appcache and merely open the app, the appcache is checked and I see the following log : E/GeckoConsole( 110): Offline cache update done, URL=http://www.everlong.org/mozilla/selfupdatinghosted/manifest.appcache I dont have an appcache manifest in my html file. So this means that we trigger "normal" appcache behaviour as soon a we have an appcache_path in the webapp manifest ? Is that wanted ?
(In reply to Julien Wajsberg [:julienw] from comment #27) > BTW, when I change the appcache and merely open the app, the appcache is > checked and I see the following log : > > E/GeckoConsole( 110): Offline cache update done, > URL=http://www.everlong.org/mozilla/selfupdatinghosted/manifest.appcache > > I dont have an appcache manifest in my html file. So this means that we > trigger "normal" appcache behaviour as soon a we have an appcache_path in > the webapp manifest ? Is that wanted ? That's what I needed. This indicates we perform the update correctly, and to answer your question, yes, based on appcache_path in OWA manifest. If you don't see the updated content after this line shows in the error console and after you reload (in this order), then it would be quit strange.
I don't see the updated content. :) I don't see anything in the apache logs either (beside the requests to manifest.appcache). And as I said before, I suspect this is because the index.html file is cached because of default heuristics. I believe it shouldn't be cached unless there are explicit cache headers but still. I don't know how to verify this though.
Got it! You've added .htaccess to affect the index, right? Not to affect the cache manifest. Then this is clearly a dupe of bug 830588. I've just finished a patch with an automated test for it!
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → DUPLICATE
Keywords: dev-doc-needed
Nope, I have no .htaccess at all, so no cache HTTP headers :) When I tried to add a .htaccess with an instant expires, then it worked as intended. If your patch affects both appcache manifest and ressources loading, than ok for duping :)
For what it's worth, I think I completely misunderstood this bug. So feel free to reopen and/or renominate if you still think something is broken (and isn't a dupe).
sicking> and the bug was really the intersection between bad logs from Gecko (will be addressed in Bug 835079) and bad headers from my server. So I'm ok with keeping it like that.
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.