834285 - we sometimes don't check for appcache manifest update if it send no http cache headers

Status: RESOLVED WONTFIX

(Core Graveyard :: DOM: Apps, defect)

Description

[Julien Wajsberg [:julienw]]

STR:
* install a hosted+appcache app
* change the appcache manifest
* check for an update

Expected:
* a download available event should be triggerred

Actual:
* nothing happens

I checked on my Apache logs that the appcache manifest is not even checked so no wonder.

Comment 1

[Julien Wajsberg [:julienw]]

Oh, I just saw that the appcache manifest is not checked even if the webapp manifest is changed.

I have the patches for bugs 831644, 832408, 833587 in my tree, I'll try a clean tree to see if this comes from one of these.

Comment 2

[Jason Smith [:jsmith]]

Could have sworn this was working before. Definitely a regression.

Comment 3

[Julien Wajsberg [:julienw]]

same thinh with a clean tree.

I'm taking this and see if I can figure it out.

Comment 4

[Jason Smith [:jsmith]]

Works for me on b2g18. Can you give a test case demonstrating this?

Comment 5

[Julien Wajsberg [:julienw]]

I use my app in http://everlong.org/mozilla/selfupdatinghosted/ but it needs manual manipulation. I can put it on github if you want ?

Comment 6

[Julien Wajsberg [:julienw]]

I don't see it anymore now. Removing the nomination until I see clearer.

Comment 7

[Jason Smith [:jsmith]]

If you can't reproduce, then let's close this.

Comment 8

[Julien Wajsberg [:julienw]]

I actually saw this so please keep it for now.

Comment 9

[Julien Wajsberg [:julienw]]

Just got it again when trying things for Bug 834338.

Here is the Webapps.jsm log:

I/Gecko   (  110): -*-*- Webapps.jsm : checkForUpdate for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp
I/Gecko   (  110): -*-*- Webapps.jsm : adding manifest etag:"4d40c404ad656"
I/Gecko   (  110): -*-*- Webapps.jsm : Checking manifest at http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp
I/Gecko   (  110): -*-*- Webapps.jsm : Got http status=304 for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp
I/Gecko   (  110): -*-*- Webapps.jsm : updateHostedApp http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp
I/Gecko   (  110): -*-*- Webapps.jsm : Saving /data/local/webapps/webapps.json
I/Gecko   (  110): -*-*- Webapps.jsm : updateHostedApp: updateSvc.checkForUpdate for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.appcache
I/Gecko   (  110): -*-*- Webapps.jsm : updateHostedApp: updateSvc.checkForUpdate return for http://www.everlong.org/mozilla/selfupdatinghosted/manifest.webapp - event is offline-cache-update-unavailable
I/Gecko   (  110): -*-*- Webapps.jsm : Saving /data/local/webapps/webapps.json
E/GeckoConsole( 1670): Content JS LOG at http://www.everlong.org/mozilla/selfupdatinghosted/js/install.js:62 in handleDownloadApplied: download applied
E/GeckoConsole( 1670): Content JS LOG at http://www.everlong.org/mozilla/selfupdatinghosted/js/install.js:67 in handleDownloadApplied: no download is available


The two last logs are from my application, it means that we get the downloadapplied event on the app.

This log shows that we're correctly doing the check in Gecko but I'm still not seeing this in my server logs.

I wonder if there could be some HTTP cache kicking in here. I have no specific expires headers so maybe some heuristic are used as for other files ?

Here are the complete headers for this appcache file :

julien@tangkou:~$ curl -I http://www.everlong.org/mozilla/selfupdatinghosted/manifest.appcache
HTTP/1.1 200 OK
Date: Fri, 25 Jan 2013 22:02:43 GMT
Server: Apache/2.2.16 (Debian)
Last-Modified: Fri, 25 Jan 2013 21:58:28 GMT
ETag: "4d42407067357"
Accept-Ranges: bytes
Content-Length: 88
Content-Type: text/plain

We see that the content-type is (incorrectly) text/plain, so maybe the heuristics for text/plain are used ?

Honza, what do you think ?

Comment 10

[Julien Wajsberg [:julienw]]

marking duplicate just as Bug 834338

Comment 11

[Julien Wajsberg [:julienw]]

Reopening because the behaviour for OWA updates is different than for appcache-manifest-in-html-tag updates.

My STR:
* install the app in http://everlong.org/mozilla/selfupdatinghosted/
* change the appcache manifest (and the webapp manifest if you want, it doesn't change anything)
* check for update

Expected:
* we should have an update

Actual:
* we don't
* I observed in Apache logs that I only see the conditional request for the webapp manifest. I also snooped into the network traffic using ngrep and saw nothing except this request. So, no request for the appcache manifest.

I've observed that it works after some time, so I suspect this is a case of heuristic caching. However, we shouldn't do an heuristic caching here because the appcache manifest is served with an ETag. Here are the complete headers for this file :

$ curl -I http://everlong.org/mozilla/selfupdatinghosted/manifest.appcache
HTTP/1.1 200 OK
Date: Wed, 30 Jan 2013 10:36:31 GMT
Server: Apache/2.2.16 (Debian)
Last-Modified: Wed, 30 Jan 2013 10:32:40 GMT
ETag: "4d47f07a284b5"
Accept-Ranges: bytes
Content-Length: 88
Content-Type: text/plain

I also observed that everything works fine with the application in http://everlong.org/mozilla/selfupdatinghostednocache/ which has an .htaccess to force an update. Here are the headers for this application's manifest :

$ curl -I http://everlong.org/mozilla/selfupdatinghostednocache/manifest.appcache
HTTP/1.1 200 OK
Date: Wed, 30 Jan 2013 10:42:12 GMT
Server: Apache/2.2.16 (Debian)
Last-Modified: Fri, 25 Jan 2013 22:21:46 GMT
ETag: "4d4245a5c8e96"
Accept-Ranges: bytes
Content-Length: 88
Cache-Control: max-age=0
Expires: Wed, 30 Jan 2013 10:42:12 GMT
Content-Type: text/plain

I also tried a pure webapp-with-appcache (I mean, not a OWA) in http://everlong.org/mozilla/appcache/, and we _don't_ check for an appcache update (after the initial download). However, Desktop Firefox (20) _does_ check for an appcache update in the same situation.

So now I'm quite sure the problem is not in our OWA update mechanism but in the actual appcache manifest update mechanism in Firefox OS.

Comment 12

[Julien Wajsberg [:julienw]]

(In reply to Julien Wajsberg [:julienw] from comment #11)
> 
> I also tried a pure webapp-with-appcache (I mean, not a OWA) in
> http://everlong.org/mozilla/appcache/, and we _don't_ check for an appcache
> update (after the initial download). However, Desktop Firefox (20) _does_
> check for an appcache update in the same situation.

I mean I tried this in Firefox OS' Browser app, which stores the appcache without user consent per Bug 835379, and with Desktop Firefox, and the behaviors are different in my case.

Comment 13

[Honza Bambas (:mayhemer)]

What is the current priority of this bug?

Comment 14

[Jason Smith [:jsmith]]

(In reply to Honza Bambas (:mayhemer) from comment #13)
> What is the current priority of this bug?

Not a priority at the moment.

Comment 15

[Guy Goldstein]

Is there a work-around for this bug? i.e. I already specify this in the .htaccess file, to no avail. The file is still cached on Firefox (not Chrome):
ExpiresByType text/cache-manifest "access plus 0 seconds"

My file is https://writerduet.com/appcache.manifest and the cache header is:
Cache-Control: no-cache

What can I specify to make Firefox not cache this file? I don't know how to test it, since after clearing my cache the bug no longer reproduces. As said above, this sounds like a Firefox caching heuristic problem.

Comment 16

[Julien Wajsberg [:julienw]]

Are you sure your .manifest file is recognized as text/cache-manifest ? On my system, files with the extension ".appcache" (eg: writerduet.appcache) have this mime type automatically. That might be your issue here.

When I use "curl -I" on your file, I see that the "Expires" header is wrong:
  Expires: Sat, 05 Jul 2014 10:20:58 GMT

So I suspect an error on your server-side.

Note that Firefox is doing what the spec says, and Chrome violates it. I personally believe that Chrome's behavior is saner though.

Comment 17

[Guy Goldstein]

Thanks for checking it out, but I still see this as a bug, and I'm not certain my planned work-around will work (can't reliably test). The cache expiration time is the same instant as it's requested, so it should never be cached, right? I'm thinking of adding these lines to the .htaccess file (for .manifest), but I'd love it if someone can confirm they'll work around the bug. Thanks!!

Header unset Cache-Control
Header unset ETag
Header set Pragma "no-cache"
Header set Expires "Wed, 11 Jan 1984 05:00:00 GMT"

Comment 18

[Julien Wajsberg [:julienw]]

I simply use this in a .htaccess file:

  ExpiresActive On
  ExpiresDefault "access"

But you need to set this only for your file. So

  ExpiresByType text/cache-manifest "access"

should definitely work provided that your file is recognized as text/cache-manifest.

You can try to rename it to "manifest.appcache" (works on my system) but maybe you can use a <Files> directive with my code instead.

Comment 19

[Sylvestre Ledru [:Sylvestre]]

Core Graveyard / DOM: Apps is inactive. Closing all bugs in this component.