233967 - Bonsai showing stale pages

Status: RESOLVED FIXED

(Webtools Graveyard :: Bonsai, defect)

Description

[Boris Zbarsky [:bzbarsky]]

The problem is as follows:

1)  Bonsai sends a last-modified header and no expires header
2)  Per HTTP spec, expires is computed by the client in this circumstance
3)  The algorithm Mozilla uses (the one recommended by the HTTP spec) is:

       now + max(Ceiling, 1/10 * (now - last_modified))

This means that pages that have not been touched for a while get an expires time
way in the future.

Now the problem is that when Mozilla is set to check pages "when out of date"
(the default setting, and the most sane one in general), it checks the current
date against expires and only requests the current last-modified header if the
page is expired.  Which means that once it's cached one of these "not touched in
a bit" bonsai pages it will be months before it actually checks whether the page
has been updated.  This makes bonsai completely unreliable.

The solution is either to send an expires header with a past date (to force
checks back to the server) or better to send on with the date of the next
scheduled update (if the updates are done via cron or something).  Or to stop
sending last-modified.

Comment 1

[Boris Zbarsky [:bzbarsky]]

>        now + max(Ceiling, 1/10 * (now - last_modified))

I meant min(), of course.....

Comment 2

[timeless]

Attachment: send expires (now+2hours)

Comment 3

[Boris Zbarsky [:bzbarsky]]

See bug 233968 comment 2.  I would say a 15-20 minute expiration would be about
right for bonsai....

Comment 4

[Christian Reis]

Comment on attachment 141562 [details] [diff] [review]
send expires (now+2hours)

Code looks correct; adjust the times to what bz suggests before checking in.

Comment 5

[Christian Reis]

The real solution would probably using a marker in the database or a
user-configurable timing,  of course..