591659 - Figure out a way to invalidate cached copies of (JS) files for releases

Status: RESOLVED FIXED

(Camino Graveyard :: Product Site, defect)

Description

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

We've known for some time that there are "over-caching" issues with files on the website, particularly acute during releases.  As we control more and more of the site with JavaScript (e.g., the UA messages, the download links), more and more content is showing stale info to new releases, which means the big buttons will link people to the last release, and the UA messages will "thank you for testing" (bug 591517) and whatnot.

This happens to all browsers (we've seen it in Camino and Chrome), and it's particularly acute for users who keep cbo/start as their home page, since those files are always cached.  Based on the comments in bug 591517, it looks like Camino will notice the new files after about 24 hours, but it would be much better if we could force them (and maybe home/start/etc., in necessary) to be invalidated when we push site changes at release.

Comment 1

[Stuart Morgan]

Should we start versioning all of our JS filenames? I know that's one fairly common way of solving this problem.

Comment 2

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to comment #1)
> Should we start versioning all of our JS filenames? I know that's one fairly
> common way of solving this problem.

That won't help if /start/ itself is cached, though, will it?  I'm not sure whether html files are "over-cached" or just the js (and css, but that's irrelevant for this particular case) ones.

Comment 3

[philippe (part-time)]

You could add a simple query string, esp on pages like /start/

path/to/filename.js?r12345678

? + 1 character is more than enough in most cases.

Otherwise, the mod_expires Apache Module could help
http://httpd.apache.org/docs/2.2/mod/mod_expires.html

Comment 4

[philippe (part-time)]

something like this in htaccess (or virtual host config file)

# enable expirations
ExpiresActive On
# expire images after a month in the client's cache
ExpiresByType image/gif A2678400
ExpiresByType image/png A2678400
ExpiresByType image/jpeg A2678400
ExpiresByType text/javascript A3600

this caches the images for longer than one month, the js file(s) for 1 hour.

I think this should work fine with Apache 1.3 (use by cb.o), but it has been a while since I worked with that

Comment 5

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

http://www.ircache.net/cgi-bin/cacheability.py?query=http%3a//caminobrowser.org/js/versions.js&descend=on
http://redbot.org/?uri=http%3A%2F%2Fcaminobrowser.org%2Fjs%2Fversions.js

http://www.ircache.net/cgi-bin/cacheability.py?query=http%3a//caminobrowser.org/js/flash-check.js&descend=on
http://redbot.org/?uri=http%3A%2F%2Fcaminobrowser.org%2Fjs%2Fflash-check.js

I'd be interested in seeing how comment 4 affects our connections/load.

If I'm understanding http://www.mnot.net/cache_docs/#CACHE-CONTROL right, no-cache might be something useful for our js (the explanation I think implies that it doesn not do what you might think its name means).

Comment 6

[philippe (part-time)]

You could limit the no-caching/cache-control to the one js file that is causing the issue here
<Files filename.js>
Header append Cache-Control "max-age=1800, must-revalidate"
</Files>
(I need to check if 'Files' takes a comma separated list)

Comment 7

[philippe (part-time)]

(In reply to comment #6)

> <Files filename.js>
> Header append Cache-Control "max-age=1800, must-revalidate"
> </Files>
> (I need to check if 'Files' takes a comma separated list)
it has to be a separate entry for each file
<Files filename.js>
...
</Files>
<Files otherfilename.js>
</Files>

Comment 8

[philippe (part-time)]

Attachment: small test folder

test case containing everything needed for a quick test, including a readme file

Comment 9

[philippe (part-time)]

Summary from IRC
Smokey did a quick check with the test case in comment 8. That failed on cb.o with a HTTP/1.1 500 Internal Server Error.

the alternate
<FilesMatch 'flash-check.js'>
Header append Cache-Control "no-cache, must-revalidate"
</FilesMatch>
also failed :-(.

That probably means that the mod_headers module is not turned on on the server.

Comment 10

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

[2:28pm] ss: I can turn on that module, but we should just wait. We're going to get Apache 2 soon enough.
[2:28pm] ardissone: which will have the module?
[2:30pm] ss: Both probably
[2:31pm] ss: But we should wait anyway
[2:31pm] ss: Because it might get turned off accidentally when moving to Apache 2.

Comment 11

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

For reference:

# Prevent serving stale JS files (uncomment on the new server only)
#<Files versions.js>
#Header append Cache-Control "no-cache, must-revalidate"
#</Files>
#<Files ua-messages.js>
#Header append Cache-Control "no-cache, must-revalidate"
#</Files>
#<Files ua-messages-start.js>
#Header append Cache-Control "no-cache, must-revalidate"
#</Files>
#<Files flash-check.js>
#Header append Cache-Control "no-cache, must-revalidate"
#</Files>

We'll uncomment those once the new server is live (to be tracked via bug 611573, if I understood Sam correctly).