Closed Bug 277813 Opened 15 years ago Closed 4 years ago

Auto-generated Expires date set too far into future

Categories

(Core :: Networking: Cache, defect)

x86
Windows 2000
defect
Not set

Tracking

()

RESOLVED FIXED
mozilla47
Tracking Status
firefox47 --- fixed

People

(Reporter: mozilla, Assigned: mcmanus)

References

()

Details

(Whiteboard: [necko-active])

Attachments

(2 files)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

[Note: This is reproducable in both Mozilla 1.7.5 and Firefox 1.0.]

On pages that return a Last-Modified header but no Expires header, Mozilla will
automatically generate its own Expires date.

Problem is, if you're viewing a page that hasn't been modified in months/years,
the auto-generated date is set far into the future.

Here's an example URL:
http://cr.yp.to/daemontools/readproctitle.html

I visited this page today (January 10, 2005). If I look at the Page Info I see:

Modified: Monday, July 16, 2001 12:33:03 AM
Expires: Wednesday, May 18, 2005 2:44:58 AM

Note how the Expires date it has generated is over 4 months (!) into the future.
This means that if, for example, this page happens to be updated next month, and
I come back, I won't see the changes because the cache isn't going to check for
changes until May.

Isn't that excessive? Shouldn't there be an upper bounds on how far into the
future expiration dates are set (e.g. 1 day)?

Reproducible: Always

Steps to Reproduce:
1. Visit http://cr.yp.to/daemontools/readproctitle.html
2. Go to Page Info, and look at the "Expires" date.
Actual Results:  
Wednesday, May 18, 2005 2:44:58 AM

Expected Results:  
Something sooner, e.g. January 11, 2005
if the page hasn't been modified in 3.5 years, I don't think it's unreasonable
to assume it won't change in the next 4 months...

and expiring it on the next day already seems _way_ too early to me.
Assignee: general → darin
Component: General → Networking: HTTP
Product: Mozilla Application Suite → Core
QA Contact: general → networking.http
Version: unspecified → 1.7 Branch
We follow RFC 2616's recommendation.  From section 13.2.4:

   Also, if the response does have a Last-Modified time, the heuristic
   expiration value SHOULD be no more than some fraction of the interval
   since that time. A typical setting of this fraction might be 10%.
That said, a hard upper limit on freshness lifetime is not unreasonable.  We
could even make it pref controlled for extra fun.  BTW, you can force the
browser to validate each cached document at least once per session by setting
the following pref:

  browser.cache.check_doc_frequency = 0

That may help if you visit a lot of sites like this.
> if the page hasn't been modified in 3.5 years, I don't think it's unreasonable
> to assume it won't change in the next 4 months...

Okay, granted, it's somewhat of an extreme example.

However, this problem isn't limited to pages that haven't been updated in years;
I've found it to be a problem for pages updated as often as once a month. I've
gotten many complaints from Firefox users about it.

As I understand it, to get the freshness lifetime, the current code subtracts
the Last-Modified date from the current date, then divides the result by 10.
Thus, if one visits a page that was last modified one month ago, the Expires
date will be set 3 days in the future.

But what if the page happens to be updated the next day? Firefox users will have
to wait upwards of _3 days_ to see the update, unless they're savvy enough to
hit Refresh in their browser (and in many cases they'd have to be psychic to
even know there's an update waiting!). Internet Explorer users, however, will
see the update just fine (as IE by default checks for updates once per session).
Component: Networking: HTTP → Cmd-line Features
Version: 1.7 Branch → 1.0 Branch
IE checks automatically.  It's once per session option is not the default.  RFC
2616 gives origin servers plenty of ways to control the freshness lifetime for a
page.  They can set "Cache-control: max-age=86400" to force the user agent to
revalidate the page once per day.  Perhaps Apache and other popular servers
should default to setting a "reasonable" max-age value for all content instead
of relying on user agents to guess what they mean.
> IE checks automatically.

BTW, I really don't know what they mean in their preferences by "automatically",
but I presume it means follow the RFC, given that once-per-session is a separate
option.
(In reply to comment #3)
> BTW, you can force the
> browser to validate each cached document at least once per session by setting
> the following pref:
>   browser.cache.check_doc_frequency = 0
> That may help if you visit a lot of sites like this.

I actually tried that setting, but it did not have any effect. It still would
not issue any new requests to the web server after closing & re-opening the
browser. (I assume that's what "once per session" is supposed to mean?)
#221036 ("Cache "once per session" not honoured") seems to cover this specific
problem.
(In reply to comment #6)
> > IE checks automatically.
> 
> BTW, I really don't know what they mean in their preferences by "automatically",
> but I presume it means follow the RFC, given that once-per-session is a separate
> option.

Actually, both "Automatically" and "Every time you start Internet Explorer"
check for updates to pages at the start of each new session. (Pages that
previously returned a Last-Modified header and no Expires header, that is.)

According to http://www.jsiinc.com/SUBF/TIP2800/rh2895.htm , "Automatically" is
an extension of "Every time you start Internet Explorer" that affects whether
the browser checks for new _images_ at the start of each session.

If Firefox(/Mozilla) could check for updates to pages at the start of every
session as IE does, that would really be ideal...
Component: Cmd-line Features → Networking: HTTP
Version: 1.0 Branch → 1.7 Branch
This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/
(In reply to comment #9)
> This is an automated message, with ID "auto-resolve01".

Same behavior exists in Firefox 1.5 Beta 1.
In case it's at all helpful, here's a patch I came up with a while back that
imposes a (hard-coded) upper limit on how far into the future auto-generated
Expires dates can be.
-> default owner
Assignee: darin → nobody
Component: Networking: HTTP → Networking
QA Contact: networking.http → networking
Version: 1.7 Branch → Trunk
Confirming, based on many bugs listed in Bug 328605.
Status: UNCONFIRMED → NEW
Component: Networking → Networking: Cache
Ever confirmed: true
QA Contact: networking → networking.cache
FWIW, Squid has a similar mechanism for capping heuristic freshness.
Duplicate of this bug: 328605
Assignee: nobody → mcmanus
Whiteboard: [necko-active]
the default squid rule seems to place a max of 3 days.. seems as good as anything for our heuristic cap.
well, lets round to a week.
Attachment #8712354 - Flags: review?(honzab.moz)
Comment on attachment 8712354 [details] [diff] [review]
autogenerated expires needs max

Review of attachment 8712354 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with comments (or updates)

::: netwerk/protocol/http/nsHttpResponseHead.cpp
@@ +414,5 @@
>  //     freshnessLifetime = max_age_value
>  // <or>
>  //     freshnessLifetime = expires_value - date_value
>  // <or>
> +//     freshnessLifetime = min(1week,(date_value - last_modified_value) * 0.10)

one_week :)) because it looks like "LWEEK"

@@ +462,5 @@
>          LOG(("last-modified = %u, date = %u\n", date2, date));
>          if (date2 <= date) {
>              // this only makes sense if last-modified is actually in the past
>              *result = (date - date2) / 10;
> +            const uint32_t kOneWeek = 60 * 60 * 24 * 3;

isn't it 3 days..?
Attachment #8712354 - Flags: review?(honzab.moz) → review+

> > +            const uint32_t kOneWeek = 60 * 60 * 24 * 3;
> 
> isn't it 3 days..?

you can tell I waffled on that :)
https://hg.mozilla.org/mozilla-central/rev/5c871eddec49
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla47
You need to log in before you can comment on or make changes to this bug.