866211 - Change the Slow Startup Tip Trigger Value

Status: RESOLVED INCOMPLETE

(Firefox Health Report Graveyard :: Web: Health Report, defect, P5)

Description

["Saptarshi Guha[:joy]"]

This bug was based on some Nightly documents before it FHR went out.

https://bugzilla.mozilla.org/show_bug.cgi?id=849879#c21

With newer data, I've updated those numbers

We have ~8MM documents/users, more than 7MM beta,rest Nightly+Aurora.

The numbers are now slightly different
(A)
For All(all prerelease), the trigger value is now 32.8  seconds,
For Nightly, the trigger value is 22.04 seconds
For Beta, the trigger is 33.3 seconds
For Aurora it's 33 seconds.

Note:

if we stick to 20 seconds as the trigger(which is the current value) then
(B)
For All(all pre-release), 20 seconds will notify 19%
For Nightly, 11.6%
For aurora: 19.5%
For beta: 19%

So only Nightly is least unchanged if we stay with 20 seconds, but others will be much so. I would recommend if it doesn't cause a delay in the delivery (and requires small effort) to make channel specific triggers given in (A)

Comment 1

[Gregory Szorc [:gps]]

CC'ing some Firefox people who implemented the slow start notification in Firefox proper. I can't remember what we're using for slow start there...

Comment 2

[:Gavin Sharp [email: [e-mail]]]

60 seconds (pref("browser.slowStartup.timeThreshold", 60000)) at the moment.

Comment 3

[Mike Connor [:mconnor]]

I'm not actually sure what we would lose by displaying the tip to more users.  If your average startup is more than 20 seconds you are having a poor experience and FHR should help with that.  If anything, I would rather be even more aggressive for when we display this tip.

Comment 4

["Saptarshi Guha[:joy]"]

Oh, i'm not so worried about showing to more users but using the same rule for all channels.

Comment 5

[Jared Wein [:jaws] (please needinfo? me)]

(In reply to Mike Connor [:mconnor] from comment #3)
> I'm not actually sure what we would lose by displaying the tip to more
> users.  If your average startup is more than 20 seconds you are having a
> poor experience and FHR should help with that.  If anything, I would rather
> be even more aggressive for when we display this tip.

Yes, this is the plan, but as we first get it rolled out we want to make sure that we don't have many false alarms.

I think we will first need to tweak our signaling here, because we currently use an average and our FHR data can have some very strange numbers in it (my FHR shows a 25,000 second startup?!).

Comment 6

[Justin Dolske [:Dolske]]

Bug 848460 has related discussion.

Comment 7

[Mike Connor [:mconnor]]

Dumping from discussion on IRC:

I think we should have three thresholds:

P75 > X: if three or more of your startups are pathologically bad, fix that problem.

Median > Y: if your typical startups are poor, fix that problem.

P75 > 15s: if your typical startups are that slow, something is wrong, or your computer is _ancient_, and we should be telling you how to make it better.

Jared, I think the proactive notification can and should be a bit more conservative, but if a user is looking for areas to improve we can be more aggressive.  The outlier sessions are a known issue, we'll have code to ignore obvious outliers before we ship.

Comment 8

[Firefox Bug Husbandry Bot]

See https://bugzilla.mozilla.org/show_bug.cgi?id=1497137; component deprecated.