571526 - ADU daily report should not include OOPP

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Austin King [:ozten]]

Currently the new daily_crashes table includes OOPP crash reports.

It should not as there is no visible crash to the end user.

OOPP - (hangid IS NULL AND process_type IS NOT NULL)

Update the SQL in the cron to remove these types of crashes from the 'Any' bucket.
Delete all rows in daily_crashes and rebuild the table.

Comment 1

[chris hofmann]

It might make it a bit more clear if the query looks like

   hangid is NULL AND process_type is NULL

that should get us firefox only crashes.  At some point we might also want to track plugin crashes on this chart too, and that query would look like

   hangid is NULL AND process_type IS plugin

Comment 2

[chris hofmann]

ozten,

I think events of the last few days show that we really nee to track all these things:

just firefox crashes
firefox + plugin crashes
firefox + plugin crashes + uniq hang pairs

something like

https://bug571118.bugzilla.mozilla.org/attachment.cgi?id=450855

helps to draw the best picture of what is going on.

maybe a combo box is the bast way of thinking about how to add and remove lines from the chart.

this is going to require extra work so maybe we spin this request off into another bug and just keep this one focused on the quicker fix to change the calculation to isolate just the firefox crashes.

Comment 3

[Austin King [:ozten]]

Filed Bug#571696.

Comment 4

[Austin King [:ozten]]

Updated Query in dev...
Reran on copy of prod data up through 6/10
http://skitch.com/oztenskitch/d87gb/crashes-per-active-daily-user-for-firefox
http://skitch.com/oztenskitch/d87gx/crashes-per-active-daily-user-for-firefox

You can use this at (requires VPN to MPT):
http://aking.khan.mozilla.org/reporter/daily?form_selection=by_version&p=Firefox&v[]=3.6.4&throttle[]=100&v[]=3.6.3&throttle[]=100&v[]=&throttle[]=100&v[]=&throttle[]=100&hang_type=crash&os[]=Windows&os[]=Mac&os[]=Linux&date_start=2010-05-28&date_end=2010-06-10&submit=Generate

Comment 5

[Austin King [:ozten]]

Sending        scripts/startDailyCrash.py
Sending        socorro/cron/daily_crash.py
Transmitting file data ..
Committed revision 2152.

Filed Bug#571960 to stage this change.

Comment 6

[Austin King [:ozten]]

This is staged, please confirm this is the correct fix.

Any - includes browser crashes, browser hangs, and OOPP hangs
Crash - includes browser crashes (no OOPP)
Hang - includes browser hangs and OOPP hangs

http://crash-stats.stage.mozilla.com/daily?form_selection=by_version&p=Firefox&v[]=3.6&throttle[]=100&v[]=3.6.3&throttle[]=100&v[]=3.6.4&throttle[]=100&v[]=&throttle[]=100&hang_type=any&os[]=Windows&os[]=Mac&os[]=Linux&date_start=2010-05-31&date_end=2010-06-14&submit=Generate

Also: When do you want this pushed to production? it will make online report unavailable for 18 hours while the table is rebuilt.

Comment 7

[chris hofmann]

> Any - includes browser crashes, browser hangs, and OOPP hangs
> Crash - includes browser crashes (no OOPP)
> Hang - includes browser hangs and OOPP hangs

I guess we should have looked closer at each of the calculations.  I think we want "Any" to reflect the combination of events that either cause the browser to be restarted or the plugin to be reloaded on the page.

It would be better and more accurate if "Any" was the combination of:
  browser crashes  --  hangid is NULL AND process_type is NULL
+ plugin crashes   --  hangid is NULL AND process_type is plugin
+ hang count (which could be represented by either the plugin side or the browser side of the hang pair) e.g  hangid is NOT NULL and process_type is plugin

This under counts hangs by a small margin and over counts places where duplicate reports are happening, but for now its the simplest method for getting to a pretty good approximation.  A slightly more accurate hang number is to look at all the hangid pairs and count a single instance of the pairs, single instance of the dups, and a single instance where the plugin or firefox side of the hang pair is missing.  Use this later method if you can find a way to make it not to processing intensive. I'm using hangid is NOT NULL then | sort | uniq -c | wc -l when processing the .csv's to get this number. 

You can see comparison numbers in the reports I'm putting up daily at http://people.mozilla.com/~chofmann/crash-stats/20100614/crash-counts.txt

Since we are close to 3.6.4 shipping it might be better to hold off on the push to production if that means we might also loose 3.6.3 and other comparison data.

I'm able to work around using the data from the .csv files until we get this on-line.

Comment 8

[Austin King [:ozten]]

> want "Any" to reflect the combination of events that either cause the browser
> to be restarted or the *plugin to be reloaded on the page*.

So Friday I thought we said an OOPP shouldn't be counted, since it didn't crash the browser.

> It would be better and more accurate if "Any" was the combination of:
>   browser crashes  --  hangid is NULL AND process_type is NULL
> + plugin crashes   --  hangid is NULL AND process_type is plugin
> + hang count (which could be represented by either the plugin side or the
> browser side of the hang pair) e.g  hangid is NOT NULL and process_type is
> plugin

I thought this bug was to correct the query until we can go back and touch the UI to breakdown hangs. Bug#571696 and Bug#565482.

> Since we are close to 3.6.4 shipping it might be better to hold off on the 
> push
> to production if that means we might also loose 3.6.3 and other comparison
> data.

Will do. Moving to 1.8.

Comment 9

[chris hofmann]

(In reply to comment #8)
> > want "Any" to reflect the combination of events that either cause the browser
> > to be restarted or the *plugin to be reloaded on the page*.
> 
> So Friday I thought we said an OOPP shouldn't be counted, since it 
> didn't crash the browser.
> 

both hangs and plugin crashes have the exact same effect from the user perspective.   they both cause the plugin to be reloaded.

so "ALL"  is the cases where the browser crashes + the places where the user sees the need to restart the plugin.

> > It would be better and more accurate if "Any" was the combination of:
> >   browser crashes  --  hangid is NULL AND process_type is NULL
> > + plugin crashes   --  hangid is NULL AND process_type is plugin
> > + hang count (which could be represented by either the plugin side or the
> > browser side of the hang pair) e.g  hangid is NOT NULL and process_type is
> > plugin
> 
> I thought this bug was to correct the query until we can go back and touch the
> UI to breakdown hangs. Bug#571696 and Bug#565482.
> 

I guess there are two things.  Making sure the counts are correct for the UI elements that we already have, and second is to make sure we can show all the values including firefox crashes, plugin crashes, and number of hangs observed by users which require some UI changes.

There is some overlap in these two tasks but this bug can continue to be getting the values that we showing reflecting the right numbers.  The other bugs can be about breaking out plugin crashes which we don't have UI for yet.

> > Since we are close to 3.6.4 shipping it might be better to hold off on the 
> > push
> > to production if that means we might also loose 3.6.3 and other comparison
> > data.
> 
> Will do. Moving to 1.8.