98884 - Automatic page refresh for "Expires" META tag not working

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect, P2)

Description

[Al]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.3+) Gecko/20010907
BuildID:    2001090703

Preferences/Advanced/Cache/"compare page with page on network" is set for
"Automatic"

The page has a META  tag:  "<META http-equiv="Expires" content="Tue, 1 Nov 2000
06:30:00 GMT">"

The page should be automatically refreshed according to W3C HTML  standard.  

Reproducible: Always
Steps to Reproduce:
1.Open any page with an Expires META tag whose "content" is prior to  the
current date.
2.
3.

Actual Results:  Doesn't automatically refresh.

Requires manual refresh.

Comment 1

[Boris Zbarsky [:bzbarsky]]

Um... I think you are confusing the Expires: and Refresh: headers....  Expires:
just tells the browser not to cache the page if the date is before the present
date....

Comment 2

[Markus Gerstel]

http://www.w3.org/TR/html4/struct/global.html#h-7.4.4.2 says :

The following sample META declaration:

<META http-equiv="Expires" content="Tue, 20 Aug 1996 14:25:27 GMT">

will result in the HTTP header:

Expires: Tue, 20 Aug 1996 14:25:27 GMT

This can be used by caches to determine when to fetch a fresh copy of the
associated document.


Can. Not should.
-> Networking:Cache anyway

Comment 3

[Al]

As I understand it, this tag is used should force the browser to fetch a fresh
copy of the document even though it may have a copy in its cache, assuming the
date has expired.  

This is useful if a document is essentially the same [e.g., only minor changes
have been made] and the author wants to force visiting browsers to fetch the
fresh copy and not use the one it may have in its cache.  

Correct me if I'm wrong.  And if so, how would an author force a browser to
fetch a fresh copy.  

Furthermore, Mozilla does not [as best I can tell] refresh the document in its
cache if the date has expired.  

Incidentally, IE5x, NS4x and IE6 all honor the tag as I have described. 

Comment 4

[gordon]

Sending upstream to HTTP.

Comment 5

[timeless]

<q class=ot>so i should be able to create an infinite loop by writing a page 
that says it expired yesterday? this sounds really stupid.</q>

Comment 6

[Al]

Timeless:

Using the "expires" tag is common practice for websites that have semi-
permanent pages with links to them.  Authors don't want to have to change 
referring links just because minor changes have been made.

There is no endless loop condition.  If the "expires" date for the cached 
document indicates the doc has expired, the browser simply fetches the one from 
the site, renders it, and replaces the one in its cache.  

If you know of an alternative way to accomplish the same thing, please inform 
me.  

As I said previously, IE6, IE5x, and NS4x all act as I stated.  If in doubt, 
try it yourself.  

Comment 7

[Bradley Baetz (:bbaetz)]

That quote from the std just says that it should be treated as an Expires:
header. Theres no reloading involved - see rfc2696 for the expires header
semantics.

darin was working on getting no-cache working - is this now working as well?

Comment 8

[Darin Fisher]

WORKSFORME... clicking on a link to a cached page containing a <meta> expires
tag with an expiration time in the past causes a reload from the server.

Comment 9

[Al]

build 2001091003 does not work.

I just visited a posted html doc with IE6, NS4x, and Moz 9.4. Then modified a 
few words in the doc and uploaded it to the site. 

IE6 and NS4x both autonmatically refreshed the doc.

MOZ did not.  I had to key the reload button.

Comment 10

[Al]

This bug is erratic. 
 
Based on Darin's report, I ran the test reported earlier today very carefully.

I just reran the test and MOZ did not fail this time.  

Comment 11

[Darin Fisher]

Al: please use about:cache to verify the expiration time of the specified
document, and if possible, use a packet sniffer either locally or on the server
(if possible) to verify that mozilla is indeed making a network request. 
alternatively, if you have a debug mozilla bug, then please enable HTTP logging
(see http://lxr.mozilla.org/seamonkey/source/netwerk/protocol/http/src/nsHttp.h#32)

thanks!

Comment 12

[Thomas Betz]

Shouldn't this bug be marked as a duplicate of bug 65892 (The HTML parser
ignores <META HTTP-EQUIV="Expires" ... >)?

these meta bugs (expires/no-cache) are important, alot of webcam sites use these
tags. 

An intresting note: the expires tag (also the no-cache tag) are working in the
sidebar! check out http://cam.xrs.net. watch the site for a minute (picture
refreshes all 30 seconds) it's allways the same picture. then clik on, you
guessed it, "add tab to ns6". see how the picture in the sidebar refreshes,
while the picture on the site is allways the same. 

This is tested with netscape 6.1 not mozilla.....

Comment 13

[Darin Fisher]

meta expires is ignored by NS6.1.. it should however work in mozilla 0.9.4

Comment 14

[Al]

Darin, it still does not work in 9.4.  I just tested it again.  NS4x, IE6 work; 
but not MOZ9.4 [2001091303]  Here is the cache data, again.  Appears to me you 
are not recording and handling the META expires tag properly.  

The bottom line is: Web designers use the "expires" tag to force pages to be 
reloaded.[Yes we know it is not a perfect solution, but it's the best we have 
for all browsers]

Here is the doc's tag:   
<meta http-equiv="Expires" content="Tue, 1 Nov 2000 06:30:00 GMT">
IE6 and NS4x have no problem with this, or any other doc, with META expires 
dates.

key: http://www.ridersite.org/JFK50/2001Participants.htm 
fetch count: 11
 
last fetched: 09/18/01 09:36:52
 
last modified: 09/18/01 09:23:17
 
expires: 09/18/01 10:37:15
 
Data size: 19229
 
Security: This document does not have any security info associated with it.
 

--------------------------------------------------------------------------------
Client: HTTP
 
charset: ISO-8859-1
 
request-method: GET
 
response-head: HTTP/1.1 200 OK
Content-Type: text/html
Accept-Ranges: bytes
Last-Modified: Tue, 18 Sep 2001 00:07:19 GMT
Etag: "905fdde1d53fc11:19100d"
Content-Length: 19229
Server: Microsoft-IIS/4.0
Date: Tue, 18 Sep 2001 13:18:10 GMT

 

Comment 15

[Darin Fisher]

just tried with a 2001-09-14 linux nightly build and observed a correct
expiration time, so it looks like this is working with recent nightly builds.

reporter: any chance you could download a nightly build and give it a try?

Comment 16

[Al]

Per Darin's request, I tested with 2001091803.

Same bug.  I always confirm the bug exist by making a slight modification to a 
page and then posting it to my web-host.  I then test that IE6 and NS4.78 work 
properly.

I repeat:  If you examine the disk cached files.  IE6 and NS4x both show 
the "expires" date to be as included in the META expires data tag.  MOZ does 
not.

Comment 17

[Darin Fisher]

wierd... i believe you :-)  but it really is wierd that this would only effect
windows and not linux.  i'll investigate this for 0.9.5

Comment 18

[Darin Fisher]

tever: can you try to repro this?

Comment 19

[Ashley Bischoff (blog at handcoding.com)]

Darin: So... NEW?

Comment 20

[Darin Fisher]

-> me

Comment 21

[Darin Fisher]

ok.. after reviewing Al's snapshot from about:cache, i finally understand the
problem here.  Al: the expiration time assigned to a cache entry is a time equal
to now + freshness_lifetime.  for <meta> expires with a time in the past, the
freshness_lifetime is zero.  if you notice, in your example, the last fetched
date is 09/18/01 09:36:52 and the expiration time is 09/18/01 10:37:15.  this
expiration time is actually the time when the page load completed.  the last
fetched time is when the page load started.  both of these times will be in the
past relative to the next page fetch, and hence this bug is INVALID.

Comment 22

[Al]

Darin:
 
You are missing the point.  The bug is the fact that you can make a change to 
the html code in a page and repost  it on the sever, same file name.   The file 
has an obsolete META expires tag.
 
IE5.5, IE6, NS4x always reload the page.  Mozilla does not.
 
Web designers can use this technique to encourage [it's not foolproof] browsers 
to reload the page.

Comment 23

[Darin Fisher]

please supply a sample document which does not work for you.  i have done so
myself and it always works.  that is, the file is always requested again from
the server.

Comment 24

[Darin Fisher]

-> untargeted

Comment 25

[Al]

I think I've found the problem.  It's the algorithm used for cache "automatic"
 
I just reran the test with Moz's pref/adv/cache set for "once per session" and 
it worked [as it should of course]!
 
I suggest that you may want to fix the "automatic" mode, since that is the 
default and the one most folks will use.
 
Bottom line is that Moz should reload the page when the "expires" date has 
expired, even if the pref setting is "automatic"  because web designers use 
this feature to "encourage" browsers to reload a page.  

Comment 26

[Bradley Baetz (:bbaetz)]

Al: Can you give a url which shows this problem?

Comment 27

[Darin Fisher]

the importance of this features is well understood ;-)

what i don't understand is why this doesn't work for you.  as i have said, i am
unable to reproduce the problem you are seeing.  my cache validation pref is
also set to validate automatically.  this is a bit frustrating... i would really
like to fix this bug, but not being able to reproduce it is getting in the way ;-)

Comment 28

[Al]

Here is URL [in the URL box].  Same problem with any of the pages.  

The one thing I haven't tried is to allow more time to elapse between when I 
post the modified page and when I go to it with Moz.  However, I always try IE6 
and NS4x before Moz.  So it's at least a minute or so before I get to Moz.

I'd be happy to send you the files; but I assume you have a way to easily fetch 
them, and their css file.  They all use jfk50.css

Comment 29

[Darin Fisher]

i think i understand the problem now...

Al: are you using view->page_source at all?

Comment 30

[Darin Fisher]

<meta http-equiv> tags are not honored when a page is loaded for viewsource.

do the following:

1) clear your disk cache
2) goto http://www.ridersite.org/JFK50/index.htm
3) goto
about:cache-entry?client=HTTP&sb=1&key=http://www.ridersite.org/JFK50/index.htm
4) notice that the expires time is less than now.
5) goto http://www.ridersite.org/JFK50/index.htm again
6) click view->page_source
7) return to
about:cache-entry?client=HTTP&sb=1&key=http://www.ridersite.org/JFK50/index.htm
8) notice that the expires time is now in the future

Al: does this somehow explain what you are seeing?

Comment 31

[Darin Fisher]

actually, this has nothing to do with viewsource... here's a much simpler way of
reproducing the problem:

1) goto http://www.ridersite.org/JFK50/index.htm
2) press shift reload to completely update the cached copy
3) goto the cache entry for this page
4) goto http://www.ridersite.org/JFK50/index.htm again
5) press reload
6) press the back button and notice that the expiration time is now in the future.

something wierd is happening in the reload case... i'm not sure what.

Comment 32

[Al]

Sorry, I inadvertantly answered you via the e-mail message.  So, i don't know if
you received my answer.  Here it is, just in case.

It well could since it made the "expires" date 3 hours in the future. 
 
If I modify a file and don't wait for at least 3 hours, then Moz won't bother to
reload.
 
By George,I believe you got it.  Good work. 
 
However, you might want to consider that, apparently, IE and NS4x don't change
the "expires" date in the cache. They just save whatever is in the META tag. 
This would seem to be a safer approach. 

Also, it appears that Moz saves the expires date from the META tag if it is in
the  future.  If it is obsolete, a modified and incorrect date is saved.  
 

Comment 33

[Darin Fisher]

-> 0.9.6

Comment 34

[Darin Fisher]

Attachment: v1.0 patch

Comment 35

[Darin Fisher]

the problem was that we weren't honoring META tags on 304 Not Modified responses.

Comment 36

[Darin Fisher]

Comment on attachment 56044 [details] [diff] [review]
v1.0 patch

r=gagan

Comment 37

[Doug Turner (:dougt)]

Comment on attachment 56044 [details] [diff] [review]
v1.0 patch

trusting that FinalizeCacheEntry does not need an active mTransport.

Comment 38

[Darin Fisher]

shouldn't be a problem... fixed-on-trunk

Comment 39

[Darin Fisher]

*** Bug 78551 has been marked as a duplicate of this bug. ***

Comment 40

[Al]

It is broken again.

Ignores "When the page is out of date".

Must key refresh, generally several times, to fetch current version from site.

Win 2K
Moz 2002010403

Comment 41

[Darin Fisher]

Al: the fix for this bug went in on 2001-01-18.  that means that only builds
after that date will have the fix.  therefore you should not expect that this
bug would be fixed in the build from 2001-01-04.  marking FIXED.

Comment 42

[Matthias Versen [:Matti]]

Darin:
Your comment #38:
>------- Additional Comment #38 From Darin Fisher 2001-11-03 21:37 -------
>
>shouldn't be a problem... fixed-on-trunk

your latest comment :
>Al: the fix for this bug went in on 2001-01-18.

It seems that you checked in this fix at : 2001-11-03 21:37
and the reporters build ID: Moz 2002010403

but maybe I'm only stupid :-)

Comment 43

[Darin Fisher]

doh! my bad... sorry for accusing you of mucking up the dates!

reopening...

Comment 44

[Darin Fisher]

-> 0.9.9

Comment 45

[Darin Fisher]

agreed.. linux build 2002020208 has this bug :-(

the testcase in comment #31 is reproducible.  investigating...

Comment 46

[Darin Fisher]

turns out i accidentally checked in some changes that i had lying around in my
tree along with my patch for bug 83471 that caused this regression.  the check
in was made on 12/7 as version 1.70 of nsHttpChannel.cpp.  the accidental
changes have been backed out.

marking FIXED.