The default bug view has changed. See this FAQ.

Compare startup(firstPaint, firstloaduri) for different values of STARTUP_USING_PRELOAD_TRIAL

RESOLVED FIXED in Moved to JIRA

Status

Mozilla Metrics
Data/Backend Reports
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: (dormant account), Unassigned)

Tracking

unspecified
Moved to JIRA
x86_64
Windows 7
Dependency tree / graph

Details

(Whiteboard: [Telemetry:P1])

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
+++ This bug was initially created as a clone of Bug #765850 +++

Saptashi, we need to redo this analysis filtering out warm startups.

I have one more theory, Saptashi, can you filter the data by SIMPLE_MEASURES_START?
According to https://metrics.mozilla.com/data/content/pentaho-cdf-dd/Render?solution=metrics2&path=%2Ftelemetry&file=telemetryHistogram.wcdf&bookmarkState={%22impl%22%3A%22client%22%2C%22params%22%3A{%22startDate%22%3A%222012-05-29%22%2C%22endDate%22%3A%222012-06-27%22%2C%22appVersion%22%3A%22%22%2C%22appName%22%3A%22Firefox%22%2C%22arch%22%3A%22%22%2C%22OS%22%3A%22%22%2C%22version%22%3A%22%22%2C%22channel%22%3A%22nightly%22%2C%22reason%22%3A%22idle-daily%22%2C%22appBuildID%22%3A%22%22%2C%22fromPlatformBuildID%22%3A%22%22%2C%22toPlatformBuildID%22%3A%22%22%2C%22excludeParam%22%3A%22%22%2C%22measure%22%3A%22SIMPLE_MEASURES_START%22%2C%22histogramCompareParam%22%3A%22appVersion%22%2C%22histogramVariablesParam%22%3A%22%22%2C%22platformBuildIDMode%22%3A%22LATEST%22%2C%22platformBuildIDTopCount%22%3A%2230%22%2C%22conditionsStatistic%22%3A%22%23conditionsStatistic%22%2C%22submissionsParameter%22%3A[[59459]]}}

23% of our users take >=1second before firefox code executes...Those could be 23% that do slow & cold startups. Could you redo the analysis, but filter on simpleMeasures.start > 0? I'm wondering if the fact that most startups are warm is skewing the data.

I'd also like to know the distribution of simpleMeasures.start for each value of  STARTUP_USING_PRELOAD_TRIAL.

Comment 1

5 years ago
https://metrics.mozilla.com/projects/browse/METRICS-844
Status: NEW → ASSIGNED
Target Milestone: Unreviewed → Targeted - JIRA
Btw, there were 48,762 	records for the following conditions

json$info$appName == 'Firefox'
&& json$info$OS   == 'WINNT'
&& json$info$appUpdateChannel == "nightly
&& appBuildId in (20120617 ... 20120623)
&& dateofsubmission from 20120617 ... 20120623

The quartiles of START is (raw units , i guess ms)

    0%       25%       50%       75%      100% 
    2        31       127         742 418823980 

So all starts > 0.

Better someone explains what the role of start is in this analysis?
Erm, slight modification. 4% of the data have start == 0. I'll complete this soon.
Definition of start still pending.
So as mentioned, only 4% of records have start == 0. The next higher value is 2 (see Comment 2).
With a percentage so small and a delta (2ms) so small, i would expect little difference in the analysis.

Indeed, there isn't. 

As for distribution of start for prefetch X preload (where start is now in seconds)

$preloadOff.prefetchOff
     0%      5%     25%     50%     75%     98%    100% 
  0.003   0.015   0.035   0.123   0.452  11.306 750.999 

$preloadON.prefetchOff
     0%      5%     25%     50%     75%     98%    100% 
  0.003   0.015   0.039   0.125   0.484  13.245 360.605 

$preloadOff.prefetchON
     0%      5%     25%     50%     75%     98%    100% 
  0.002   0.010   0.031   0.140   0.991  15.938 881.878 

$preloadON.prefetchON
     0%      5%     25%     50%     75%     98%    100% 
  0.002   0.010   0.031   0.126   1.007  16.691 581.385
Created attachment 641220 [details]
Start Distribution

In case the table above is difficult to decipher the figure will help.

The figure has percentiles of Log(start,2) (2%,25%,50,75,and 98). As can be seen, Prefetch = On is associated with larger values of start as compared to prefetch=Off.

Also, no interaction with prefetch and preload.
If I'm reading the data correctly, I think it says that cold startups are positively affected pretty significantly when prefetch is disabled.  

But there is no way to disable prefetch only for some startups, nor can we disable it only for a specific firefox install only.

Since most installs are hot startups, I think the backout process for disabling prefetch should continue.  Do you agree Taras?
(Reporter)

Comment 7

5 years ago
(In reply to Brian R. Bondy [:bbondy] from comment #6)
> Since most installs are hot startups, I think the backout process for
> disabling prefetch should continue.  Do you agree Taras?

I think we should turn off prefetch when START > 0.5s. For safety we can turn it back on on an interval
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
(Reporter)

Comment 8

5 years ago
Sorry, didn't mean to close this. 

Saptashi, the data in comment 4 is very handy, however we need a matching table for firstPaint/firstLoadURI to make a decision.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
For firstpaint (seconds)

$prefetchOff.preloadOff
      2%      25%      50%      75%      98% 
 0.57900  2.07050  5.82000 14.50750 80.03664 

$prefetchON.preloadOff
      2%      25%      50%      75%      98% 
 0.53100  1.56450  3.79600 10.59325 67.91190 

$prefetchOff.preloadON
      2%      25%      50%      75%      98% 
 0.64000  2.24750  5.48300 13.21050 79.81156 

$prefetchON.preloadON
      2%      25%      50%      75%      98% 
 0.54924  1.60800  3.65200  9.09600 64.25872 




For *URI (seconds)

$prefetchOff.preloadOff
      2%      25%      50%      75%      98% 
 0.44100  1.91200  5.97600 14.79150 85.87704 

$prefetchON.preloadOff
      2%      25%      50%      75%      98% 
 0.39022  1.37575  3.96100 11.04075 72.41188 

$prefetchOff.preloadON
     2%     25%     50%     75%     98% 
 0.4860  2.0910  5.5940 13.6750 82.8168 

$prefetchON.preloadON
      2%      25%      50%      75%      98% 
 0.39100  1.45200  3.77600  9.86000 69.57224
(Reporter)

Comment 10

5 years ago
 According to the 48K number above populations for last 2 columns are ~12K, 2K...So these wouldn't be swayed by a few outliers. 

Looks like priming prefetch with preload is the best way to go(ie least risk of regressing people). Thanks a lot Saptashi!
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
Great data :)

OK so the plan is to re-enable preload on Windows (always) and continue with reverting the disable prefetch task (There is only 1 patch left to land in 2 weeks). Correct?
(Reporter)

Comment 12

5 years ago
(In reply to Brian R. Bondy [:bbondy] from comment #11)

> OK so the plan is to re-enable preload on Windows (always) and continue with
> reverting the disable prefetch task (There is only 1 patch left to land in 2
> weeks). Correct?

I think that's the best short-term action based on this.
(Reporter)

Updated

5 years ago
Blocks: 774444
(Reporter)

Updated

5 years ago
Blocks: 776720
(Reporter)

Updated

4 years ago
Duplicate of this bug: 778123
You need to log in before you can comment on or make changes to this bug.