Closed Bug 600865 Opened 14 years ago Closed 12 years ago

4.0 nightly users climbing more rapidly than expected

Categories

(Mozilla Metrics :: Frontend Reports, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX
Unreviewed

People

(Reporter: dre, Assigned: dre)

Details

Attachments

(2 files)

(choffman said in bug 599828 comment #19)
> mentioned to Daniel that it looks like we might be over counting now  40k trunk
> users is a lot.  its possible the grow really looks like this due to recent
> tech press coverage on nightlies with new firefox 4.0 features, but I'd expect
> crashes to rise as we add users.  # crashes is basically flat. 
> 
>                crashes  adu    adu
>                         after  before
>                         fix    comment 16 fix
> 
> 2010-09-28    2007    43192    
>   -- reprocessed --
> 2010-09-27    1741    40509    4479
> 2010-09-26    1767    32320    4307
> 2010-09-25    1794    30232    6032
> 2010-09-24    1781    33378    8463
> 2010-09-23    1966    32737  10659
> 2010-09-22    2344    30958  15951
> 2010-09-21    2493    28803  25296
> 2010-09-20    1969    25458  25443    
>     -- previous days match up...
> 2010-09-19    1612    20031  20031
> 2010-09-18    1682    18371  18371
> 2010-09-17    2481    20792    ...
> 2010-09-16    695    18556    ...


(choffman said in bug 599828 comment #24)
> up another 3,000 users yesterday to 46,258.
> 
> it would be good if they did, but uptake graphs don't appear to track to "pre"
> versions to allow easy tracking of trunk users across "pre" versions.
> 
> digging some info out of the "breakdown by version" data on metrics it shows we
> haven't seen anything near 40k+ trunk users, or even 30k+ for a couple of
> months.
> 
> 3.0b2pre = 35,215  july 20
> 4.0b3pre = 33,470  aug 8
> 4.0b4pre = 25,555  aug 17
> 4.0b5pre = 26,859  aug 31

(beltzner said in bug 599828 comment #25)
> Looking at the graph here http://grab.by/6Dv8 it looks like we doubled our ADUs
> on the nightly 4.0 channel in June.
Attached image 4.0x "pre" migration
this might better to show the rolling migration motion and peaks for various pre versions.   both motion and peak for 7pre are unusual.
I updated the HTML stats to show 4.0b7 nightly breakdowns:
https://metrics.mozilla.com/stats/firefox.shtml

Let me know if this helps explain stuff or where we should dig in for investigation.
up to 49845 yesterday after a lull in growth over the weekend.

I'm not sure how to read https://metrics.mozilla.com/stats/firefox.shtml  there are a set unusually high numbers just off the front edge of the curve in the "Firefox 4.0b7pre nightly channel daily active installations by build number:

18757    20101002041357  day 1
15664    20101002041357  day 2
10440    20101001082844  day 3
16929    20100930041305  day 4
...
...
...

if you subtract out those numbers, or replace with something that matches a historical trend, you might get something that is a better match to the historical number of trunk users.

Does what is shown there now reflect an increasing number of users updating to a build that is two or three days old?  again, not sure how to read that chart.
Believe you are seeing the data they are asking for in bug 601859.  There was a bug in those builds that prevent people from updating.
I'm not sure thats it.  The higher than normal numbers go back much further to when the out-of-band trunk increases started.

see third (non-zero) build down in d-5, d-6, d-7, d-8, d-9, d-10

before d-10 and build 20100924040732 the numbers look more like what we have been seeing for trunk users.
I'm going to move this to a data analysis request. please let me know what information we can provide to help you explain what you are seeing.
Severity: major → enhancement
I'm not sure how this gets marked as an enhancement.

There is a strong indication from levels of crash data that we continue to have around 20k-30k trunk users.   ADU data says we have ramped up to +50,000 users since sept 20?

It seems highly likely that there either is a bug in not getting the right crash counts on trunk builds, or a bug in the way we are counting adu's.  The adu numbers appear to be the culprit since they are the departure from the normal and since they were the changing factor in the the fix for bug 599828.

date          daily              cashes 
              4.0b7pre  4.0b7pre per 100
              crashes   adus     users

2010-10-06	2261	51187	4.42%	44.1714
2010-10-05	2020	51233	3.94%	39.4277
2010-10-04	1734	49845	3.48%	34.7878
2010-10-03	1618	41193	3.93%	39.2785
2010-10-02	1666	40565	4.11%	41.0699
2010-10-01	1699	45800	3.71%	37.0961
2010-09-30	1740	46947	3.71%	37.0631
2010-09-29	2211	46258	4.78%	47.7971
2010-09-28	2007	43192	4.65%	46.4669
2010-09-27	1741	40509	4.3%	42.9781
2010-09-26	1767	32320	5.47%	54.672
2010-09-25	1794	30232	5.93%	59.3411
2010-09-24	1781	33378	5.34%	53.3585
2010-09-23	1966	32737	6.01%	60.0544
2010-09-22	2344	30958	7.57%	75.7155
2010-09-21	2493	28803	8.66%	86.5535
2010-09-20	1969	25458	7.73%	77.3431

2010-09-19	1612	20031	8.05%	80.4753
2010-09-18	1682	18371	9.16%	91.5573
2010-09-17	2481	20792	11.93%	119.325
2010-09-18	1682	18371	9.16%	91.5573
2010-09-17	2481	20792	11.93%	119.325
Don't worry about the severity being marked enhancement. That is just to flag that we are tracking this as a data analysis request.

Counting ADU is a pretty simple thing.  We just add up the number of blocklist pings every day.  You would be the expert on how we get the right crash counts on trunk builds.

bug 599828 was about a change to a parameter in the blocklist ping that caused us to reject blocklist pings from trunk builds for processing.  There isn't any way that fixing that bug could have caused us to start counting them twice or otherwise generating new requests that aren't in the logs.

As I said in comment #7, give me some ideas for what data we can provide to you regarding ADU/blocklist pings that will help get to the bottom of this and we'll do our best to provide them.
Group: metrics-private
CC list accessible: false
Not accessible to reporter
CLosing all old (created < 1-1-2011 ) cases still in NEW state.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: