924116 - HTTP cache v2: persists frecency and experiment with half-life

Status: RESOLVED FIXED

(Core :: Networking: Cache, defect)

Description

[Honza Bambas (:mayhemer)]

Now it's 90 days and it's not persisted.  So we can change it.

I personally propose to lower the decay time to some 24hours only.  The cache will need to throw files after few days of browsing and 90 days is IMO too much.  Entries that are accessed only 2 times 80 days ago will live over entries I've visit now only ones - e.g. a new page I've just discovered.

Other option is to store multiple frecency values with different decays and add telemetry for A|B|C|... testing to find out which value is most effective.  Tho, hard to say what is the quantification of efficiency here...

Comment 1

[Honza Bambas (:mayhemer)]

This is super easy actually - we can store frecency converted back to time (frecency * half-life) and on load covert back with the current half-life that might change between browser sessions.

Comment 2

[Honza Bambas (:mayhemer)]

Attachment: v1

Comment 4

[Honza Bambas (:mayhemer)]

Attachment: Prerequisity (bug 913811, v2.1, r=michal)

Comment 5

[Michal Novotny [:michal]]

Comment on attachment 8349750 [details] [diff] [review]
v1

Review of attachment 8349750 [details] [diff] [review]:
-----------------------------------------------------------------

::: netwerk/protocol/http/nsHttpChannel.cpp
@@ +93,5 @@
>      else {
>          Telemetry::Accumulate(Telemetry::HTTP_CACHE_DISPOSITION_2_V2, hitOrMiss);
> +
> +        int32_t experiment = CacheObserver::HalfLifeExperiment();
> +        if (experiment > 0 && hitOrMiss == kCacheMissed) {

Why we want to collect only miss statistics?

Comment 6

[Honza Bambas (:mayhemer)]

(In reply to Michal Novotny (:michal) from comment #5)
> Why we want to collect only miss statistics?

Other possibility is HIT.  HIT = TOTAL - MISS.  We have overall total (when we sum all HTTP_CACHE_DISPOSITION_2_V2 values).  So, I don't see a point.  If you are concerned about evenness of the 4 four test values (what I'm not) then we might also collect TOTAL cache accesses per the experiment value, but I believe those will be even.

What I'm more concerned is how we persist the experiment value in preferences and prevent it's loss because of a crash...  Thinking of a main-thread timer delayed a bit after startup.

Comment 7

[Honza Bambas (:mayhemer)]

Attachment: v1 [as landed], r=michal

https://hg.mozilla.org/mozilla-central/rev/096939370788

Comment 8

[Matt Brubeck (:mbrubeck)]

This experiment is still running several years later.  Have there been any conclusions?  Can the experiment code be removed?

Comment 9

[Honza Bambas (:mayhemer)]

(In reply to Matt Brubeck (:mbrubeck) from comment #8)
> This experiment is still running several years later.  Have there been any
> conclusions?  Can the experiment code be removed?

I think yes.  The conclusion is that the half-live has surprisingly no effect on cache hit rate.  Since the cache hit rate is similar to other browsers, I believe there is no need to fiddle with the preference.  Let's remove the code and revert the prefs to something "in the middle".

I'll file a bug.
Thanks.