346820 - allow sites to specify minimum update frequency

Status: RESOLVED WONTFIX

(Firefox Graveyard :: Microsummaries, defect, P5)

Description

[Myk Melez [:myk] [@mykmelez]]

A Yahoo! guy I talked to at OSCON last week asked if the microsummary service provided any mechanism for sites to limit how frequently the service downloaded pages to regenerate their summaries using generators.  Currently, we don't have such a mechanism.

He suggested that we implement one, so sites can defend themselves against the load from popular generators which try to refresh microsummaries too frequently.  That seems like a reasonable request, since a generator might well set its update interval setting to a very low number and then become popular, causing problems for sites that get hit with too much traffic.

Since the issue is the downloading of pages, which are virtually always HTML, perhaps we could define a meta tag (and HTTP response header equivalent) that specifies the minimum update interval.  The microsummary service would sniff that value after downloading a page and make sure the next update honors it, regardless of what the generator requests.

We would probably want the site's setting to also override the browser.microsummary.updateInterval preference (which can be set to as little as one minute) and perhaps also the hardcoded default setting (every half hour).

But if we ever enable users to set a specific, explicit update interval for individual bookmarks, then we might decide to prioritize that over the site's preferences.

Comment 1

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Darin suggests sites could do this by checking for

X-Moz: microsummary

and if present, send:

Expires: ... and/or Cache-Control: max-age=...
and
Vary: X-Moz

Comment 2

[Myk Melez [:myk] [@mykmelez]]

Ok, I'll test to make sure it works and then add this to the microsummaries documentation for web sites.

Comment 3

[Myk Melez [:myk] [@mykmelez]]

It turns out that Vary doesn't work in this case, since Vary only invalidates the cache for requests that have a different value for the given header(s).  It doesn't invalidate the cache for requests that do not include the header(s) at all.

So if a site sends "Vary: X-Moz" along with some cache control headers, those headers will apply to microsummary requests (those with "X-Moz: microsummary"), but they will also apply to most non-microsummary requests (those without an "X-Moz" header).

The only requests to which those cache control headers won't apply are those with X-Moz set to something else like "prefetch".

Nevertheless, it is possible for sites to specify a minimum update frequency for microsummary requests via cache control headers that specify when the cached response expires.  Sites should just be cognizant of the limitation that doing so will also set the cache expiration for user requests.

Comment 4

[Myk Melez [:myk] [@mykmelez]]

Darin: might it be possible to partition the cache (via namespaces, flags, or some other mechanism) so that responses to microsummary-related requests are separated from responses to user-initiated requests?  Then sites that return a different set of cache headers for microsummary-related requests could have those headers apply only to those requests.

Comment 5

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

- BUGSPAM -
Wontfixing all Microsummaries bugs, since the feature has been removed from the core product and previous versions won't get further fixes for it.
If interested in supporting Microsummaries in your add-on, you're free to use our old microsummaries code and to search all previously open bugs by looking for [microsummaries-feature-removal] in the status whiteboard field.