547269 - socorro not reporting correct number of crashes on 3.6 Crashes per ADU report

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[chris hofmann]

http://extraexploit.blogspot.com/ points out that thre report indicates large volume changes (high and low) in the crash per adu report.

Looking 
http://crash-stats.mozilla.com/daily?form_selection=by_version&p=Firefox&v[]=3.6&v[]=&v[]=&v[]=&os[]=Windows&os[]=Mac&os[]=Linux&date_start=2010-02-04&date_end=2010-02-18&submit=Generate

I get different results for the suspect days, but still not as expected.

The .csv files seem to have more reliable data that is something like we should be expecting.

day total-crashes number-of-signatures for 3.6
20100201 52568 11521
20100202 57501 12230
20100203 54787 11839
20100204 60644 12972
20100205 48740 11414
20100206 61327 13090
20100207 65058 13748
20100208 71433 14431
20100209 73494 14415
20100210 75453 14643
20100211 75434 14847
20100212 75782 14944
20100213 74119 14909
20100214 76969 15401
20100215 82800 16177

Comment 1

[Austin King [:ozten]]

Is this a duplicate of Bug#544583 ?

Comment 2

[chris hofmann]

I thought that bug was related to catching up on the backlog, and I thought we were over the backlog problems a few days ago.   Maybe aravind knows if we still have backlog processing going on.


right not the report shows  around 3.5-4.1k crashes for 3.6 on Mod/Tue/Wed. this week and that can't be right.

Date  	3.6
Crashes 	1k ADU 	Ratio
2010-02-18 	35034 	18187.285 	0.193%
2010-02-17 	4177 	17622.842 	0.024%
2010-02-16 	3793 	16899.452 	0.022%
2010-02-15 	3542 	16308.573 	0.022%
2010-02-14 	73629 	13876.292 	0.531%
2010-02-13 	74682 	13320.619 	0.561%
2010-02-12 	75506 	14962.462 	0.505%
2010-02-11 	3772 	15063.41 	0.025%
2010-02-10 	3666 	13117.153 	0.028%
2010-02-09 	3605 	14468.232 	0.025%
2010-02-08 	65594 	13977.862 	0.469%
2010-02-07 	2808 	11627.394 	0.024%
2010-02-06 	2806 	10871.585 	0.026%
2010-02-05 	60247 	11464.072 	0.526%

Comment 3

[Aravind Gottipati [:aravind]]

(In reply to comment #2)
> I thought that bug was related to catching up on the backlog, and I thought we
> were over the backlog problems a few days ago.   Maybe aravind knows if we
> still have backlog processing going on.

Nope, we are all caught up now.  Do you know which report gives you this data.  We run a few reports every night, if you can tell me which report gives you this data, I can dig through its output and see if there was something off about it.

Comment 4

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

The problem appears to be Windows only; compare:

Windows:
http://crash-stats.mozilla.com/daily?form_selection=by_version&p=Firefox&v[]=3.6
&v[]=&v[]=&v[]=&os[]=Windows&date_start=2010-01-07&date_end=2010-02-20&submit=Ge
nerate
(bizarre data)

Mac:
http://crash-stats.mozilla.com/daily?form_selection=by_version&p=Firefox&v[]=3.6&v[]=&v[]=&v[]=&os[]=Mac&date_start=2010-01-07&date_end=2010-02-20&submit=Generate
(as expected)

Linux:
http://crash-stats.mozilla.com/daily?form_selection=by_version&p=Firefox&v[]=3.6&v[]=&v[]=&v[]=&os[]=Linux&date_start=2010-01-07&date_end=2010-02-20&submit=Generate
(as expected)

Comment 5

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Bugzilla mangled (only linkified part of) the first of my three links in the previous comment.

Comment 6

[Austin King [:ozten]]

The bug I linked to is that we found an issue in our web service that causes indeterminate results.

Details:
The SQL used a column alias that was the same as an existing column (os_name).
If the rows returned had more than one row with the same os - Windows NT or something, instead of collapsing the rows it would return multiple.

Our code used the first unique os per row.

So calling the API multiple times could produce:
Windows 12 crashes
Windows 14883 crashes
Windows 14883 crashes
Windows 12 crashes

Etc.

This is fixed and being deployed to stage.

Comment 7

[chris hofmann]

you can see problem if you run this report

http://crash-stats.mozilla.com/daily?form_selection=by_version&p=Firefox&v[]=3.6&v[]=&v[]=&v[]=&os[]=Windows&os[]=Mac&os[]=Linux&date_start=2010-02-04&date_end=2010-02-18&submit=Generate

it will deliver something that looks like the output in comment 2.

we should be showing around 82800 crashes on firefox 3.6 for 2/15.  the report shows 

2010-02-15     3542

Comment 9

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

This fix was pushed last night and has resolved the issue with low crash count
numbers.  Nice work Austin.

Comment 10

[chris hofmann]

3.6 data shows up nicely, but for some reason I'm not able to get some of the other releases including 3.5.8 and 3.0.18

http://crash-stats.stage.mozilla.com/daily?form_selection=by_version&p=Firefox&v[]=3.6&throttle[]=15&v[]=3.5.8&throttle[]=.375&v[]=&throttle[]=100&v[]=&throttle[]=100&os[]=Windows&os[]=Mac&os[]=Linux&date_start=2010-02-17&date_end=2010-02-25&submit=Generate

any thoughts on why those releases generate "No crash data is available for this report." errors?

Comment 11

[Austin King [:ozten]]

(In reply to comment #10)
You'll need to add those versions via stage's admin panel.
http://crash-stats.stage.mozilla.com/admin/branch_data_sources

Note: These are available in production, since they have been added via the admin branches panel.
http://crash-stats.mozilla.com/daily?form_selection=by_version&p=Firefox&v[]=3.6&throttle[]=15&v[]=3.5.8&throttle[]=.375&v[]=&throttle[]=100&v[]=&throttle[]=100&os[]=Windows&os[]=Mac&os[]=Linux&date_start=2010-02-17&date_end=2010-02-25&submit=Generate