Last Comment Bug 659425 - get average session length from crash stats and Test Pilot studies
: get average session length from crash stats and Test Pilot studies
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: General (show other bugs)
: unspecified
: x86 Windows 7
: -- normal (vote)
: ---
Assigned To: Robert Strong [:rstrong] (use needinfo to contact me)
:
:
Mentors:
Depends on:
Blocks: 657472
  Show dependency treegraph
 
Reported: 2011-05-24 13:30 PDT by Robert Sayre
Modified: 2011-10-11 13:46 PDT (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments

Description Robert Sayre 2011-05-24 13:30:49 PDT
We need to know how often Firefox users restart their browsers.
Comment 1 Robert Strong [:rstrong] (use needinfo to contact me) 2011-05-26 00:08:17 PDT
Below is the Windows uptime data Firefox 3.6 crashes for the period of 2010-12-04 to 2011-05-10.

Besides invalid entries the report excludes the uptime data for crashes on systems where firefox.exe has been running for less than 15 minutes (there were many of these and increasing the time to 30 minutes reduced the under 12 hours percentage by over 10 percent iirc) and for more than 84 days (there were very few of these and there were a few that claimed to be running for 1000's of days) to hopefully reduce the skewing due to this data being from crashes. I also excluded crashes that didn't have a value for seconds since last crash or that had a value for seconds since last crash that was less than 7 days.

Summary
=================================
 Crash lines processed : 23353548
      Crashes included :  2636065

Skipped Line Breakdown
=======================================================
                         Invalid line format :       50
                    Invalid firefox.exe name :      170
       Less than minimum uptime (15 minutes) :  1771321
          More than maximum uptime (84 days) :     3519
                    Missing since last crash :  4666778
 Less than minimum since last crash (7 days) : 14275645

Hours |  Count  | Percent
------+---------+--------
  12  | 2243942 |  85.12%
  24  |  130711 |   4.96%
  36  |   74049 |   2.81%
  48  |   34024 |   1.29%
  60  |   30793 |   1.17%
  72  |   17793 |   0.67%
  84  |   18293 |   0.69%
  96  |   11046 |   0.42%
 108  |   11708 |   0.44%
 120  |    7387 |   0.28%
 132  |    7656 |   0.29%
 144  |    5245 |   0.20%
 156  |    5740 |   0.22%
 168  |    3990 |   0.15%
>168  |   33688 |   1.28%


This is the same report without excluding crashes that didn't have a value for seconds since last crash.

Summary
=================================
 Crash lines processed : 23353548
      Crashes included :  5689719

Skipped Line Breakdown
=======================================================
                         Invalid line format :       50
                    Invalid firefox.exe name :      399
       Less than minimum uptime (15 minutes) :  3381140
          More than maximum uptime (84 days) :     6595
 Less than minimum since last crash (7 days) : 14275645

 Hours |  Count  | Percent
-------+---------+--------
   12  | 4985179 |  87.62%
   24  |  255241 |   4.49%
   36  |  134126 |   2.36%
   48  |   60510 |   1.06%
   60  |   53418 |   0.94%
   72  |   30493 |   0.54%
   84  |   30805 |   0.54%
   96  |   18360 |   0.32%
  108  |   19214 |   0.34%
  120  |   12118 |   0.21%
  132  |   12630 |   0.22%
  144  |    8545 |   0.15%
  156  |    9318 |   0.16%
  168  |    6413 |   0.11%
 >168  |   53349 |   0.94%
Comment 2 Robert Strong [:rstrong] (use needinfo to contact me) 2011-05-26 00:13:43 PDT
I emailed jono to see if we can get uptime from existing TestPilot data and I'll be following up with him sometime in the next couple of days.

I'm going to run the report against a similar set of Firefox 4.0 data as well.
Comment 3 Robert Strong [:rstrong] (use needinfo to contact me) 2011-05-26 01:29:45 PDT
Below is the Windows uptime data Firefox 4.0 crashes for the period of 2010-11-11 to 2011-05-10.

Besides invalid entries the report excludes the uptime data for crashes on systems where firefox.exe has been running for less than 15 minutes and for more than 84 days to hopefully reduce the skewing due to this data being from crashes as the Firefox 3.6 report. I also excluded crashes that didn't have a value for seconds since last crash or that had a value for seconds since last crash that was less than 7 days.

The Firefox 4.0 report includes crashes from before the Firefox 4.0 release and also has a much larger number of nightly users than the Firefox 3.6 report. I assume that this will increase the number of crashes that happen and lessen the number of longer uptime values.

Summary
=================================
 Crash lines processed : 12124849
      Crashes included :   826424

Skipped Line Breakdown
======================================================
                         Invalid line format :      61
                    Invalid firefox.exe name :      12
       Less than minimum uptime (15 minutes) :  490072
          More than maximum uptime (84 days) :    1930
                    Missing since last crash : 2982184
 Less than minimum since last crash (7 days) : 7824166

 Hours |  Count  | Percent
-------+---------+--------
   12  |  696468 | 84.27%
   24  |   47556 |  5.75%
   36  |   26384 |  3.19%
   48  |   12280 |  1.49%
   60  |   10231 |  1.24%
   72  |    6382 |  0.77%
   84  |    5593 |  0.68%
   96  |    3714 |  0.45%
  108  |    3409 |  0.41%
  120  |    2181 |  0.26%
  132  |    2081 |  0.25%
  144  |    1413 |  0.17%
  156  |    1476 |  0.18%
  168  |    1018 |  0.12%
 >168  |    6238 |  0.75%


This is the same report without excluding crashes that didn't have a value for seconds since last crash.

Summary
=================================
 Crash lines processed : 12124849
      Crashes included :  2678407

Skipped Line Breakdown
======================================================
                         Invalid line format :      61
                    Invalid firefox.exe name :      58
       Less than minimum uptime (15 minutes) : 1617705
          More than maximum uptime (84 days) :    4452
 Less than minimum since last crash (7 days) : 7824166


        Invalid line format :      61
   Invalid firefox.exe name :      58
   Less than minimum uptime : 1617705
   More than maximum uptime :    4452
 Less than since last crash : 7824166


 Hours |  Count  | Percent
-------+---------+--------
   12  | 2392769 | 89.34%
   24  |  117365 |  4.38%
   36  |   58474 |  2.18%
   48  |   26067 |  0.97%
   60  |   21011 |  0.78%
   72  |   12361 |  0.46%
   84  |   11166 |  0.42%
   96  |    6887 |  0.26%
  108  |    6371 |  0.24%
  120  |    4136 |  0.15%
  132  |    3723 |  0.14%
  144  |    2578 |  0.10%
  156  |    2673 |  0.10%
  168  |    1816 |  0.07%
 >168  |   11010 |  0.41%
Comment 4 Robert Strong [:rstrong] (use needinfo to contact me) 2011-06-07 16:43:06 PDT
TestPilot data

20348 records which equates to approximately 10174 records. I removed the records that were less than 15 minutes which accounted for 4960 records. There were also 2 records that had a huge value for the difference between startup and shutdown that I removed. This left a total of 4532 records when records that didn't have matching startup and shutdown records were removed.


 Hours |  Count  | Percent
-------+---------+--------
   12  |   4431  | 97.77%
   24  |     69  |  1.52%
   36  |     16  |  0.35%
   48  |      5  |  0.11%
   60  |      6  |  0.13%
   72  |      1  |  0.02%
   84  |      1  |  0.02%
   96  |      1  |  0.02%
Comment 5 Nick Thomas [:nthomas] 2011-06-27 03:06:19 PDT
(In reply to comment #0)
> We need to know how often Firefox users restart their browsers.

I'm interested in knowing this too. 

(In reply to comment #4)
> 20348 records which equates to approximately 10174 records. I removed the
> records that were less than 15 minutes which accounted for 4960 records.

I'm wondering why the shorter data was excluded here. Couldn't users genuinely be starting Firefox for short periods ?
Comment 6 Nick Thomas [:nthomas] 2011-06-27 03:08:31 PDT
One more question. Does uptime from crash-stats measure the total number of seconds since the last crash ? How do you get from there to the average time Firefox was running without crashing ?
Comment 7 Robert Strong [:rstrong] (use needinfo to contact me) 2011-06-27 12:18:43 PDT
(In reply to comment #5)
> (In reply to comment #4)
> > 20348 records which equates to approximately 10174 records. I removed the
> > records that were less than 15 minutes which accounted for 4960 records.
> 
> I'm wondering why the shorter data was excluded here. Couldn't users
> genuinely be starting Firefox for short periods ?
They very well could... the data set used isn't ideal to get this type of data when using crashstats since there can be multiple crashes in a row.

(In reply to comment #6)
> One more question. Does uptime from crash-stats measure the total number of
> seconds since the last crash ?
No. It is the last crash date/time

> How do you get from there to the average time
> Firefox was running without crashing ?
The data I derived is for number of people than ran Firefox without restarting over 12 hour groups just using uptime for how long Firefox has been running.

If there is a report using this data you'd like I can most likely create one or supply you with the raw data I have.
Comment 8 Lawrence Mandel [:lmandel] (use needinfo) 2011-10-11 13:44:55 PDT
Rob, afaik this data was used in the recent change from 12 to 24 hours for display of the app update UI. Is this work complete?
Comment 9 Robert Strong [:rstrong] (use needinfo to contact me) 2011-10-11 13:46:35 PDT
yep though we *might* want to investigate deeper at some point.

Note You need to log in before you can comment on or make changes to this bug.