641484 - Document/clarify what 'Crashes Per Active Daily User' means

Status: VERIFIED FIXED

(Socorro :: General, task)

Description

[Benoit Jacob [:bjacob] (mostly away)]

The graphs on the crash-stats home page are very hard to make sense of. What does the y-axis unit mean? How is 'Active Daily User' defined? Has every user really crashed twice in average? Does that include plugin crashes (I think it shouldn't)?

Some people say it's not per user but per-100-users, is that correct?

Maybe add a FAQ also explaining: why does the number of crashes per active daily user spike shortly after a release? Why do all 3 curves move together upwards/downwards on certain days?

Comment 1

[Robert Kaiser]

I guess linking to a wiki page that describes "common trend patterns" or so would be a good idea for the latter part.

The others should added directly on the page as some sort of description to give people who don't know the intrincics knowledge what this really is about.

Comment 2

[Brandon Savage [:brandon]]

Moving onto the 1.7.8 milestone.

Comment 3

[[:jberkus] Josh Berkus]

I've documented this on the assumptions etc. wiki page, but there really needs to be something on the ADU page itself.

Comment 4

[Brandon Savage [:brandon]]

Attachment: Adding some sort of description

I used the text provided by jberkus on the assumptions page for the definition. It's placed in the search box so that users can understand the term, and effectively make decisions based on the definition. If you have a better place to put it, or improved wording, please r- and let me know what your thoughts are.

Comment 5

[Brandon Savage [:brandon]]

Fixed in revision 3134.

Comment 6

[Benoit Jacob [:bjacob] (mostly away)]

Hey! The patch here only adds a description that is certainly useful, but still the labeling of the axes of the graph shown on the crash-stats main page is not updated, and according to the clarification from your patch, it's wrong! (It should say "crashes per 100 ADUs" instead of "crashes per ADU" iiuc).

Comment 7

[Robert Helmer [:rhelmer]]

(In reply to comment #6)
> Hey! The patch here only adds a description that is certainly useful, but still
> the labeling of the axes of the graph shown on the crash-stats main page is not
> updated, and according to the clarification from your patch, it's wrong! (It
> should say "crashes per 100 ADUs" instead of "crashes per ADU" iiuc).

Reopening to cover this. We also have been discussing some open questions from comment 0:

(In reply to comment #0)
> The graphs on the crash-stats home page are very hard to make sense of. What
> does the y-axis unit mean? How is 'Active Daily User' defined? Has every user
> really crashed twice in average? Does that include plugin crashes (I think it
> shouldn't)?


It'd probably be clearer to track the plugin crash questions as a separate bug. If you want to drop by IRC (#breakpad) or post to mozilla.tools.socorro we can help figure out if this is a bug or more something for the FAQ.


> Some people say it's not per user but per-100-users, is that correct?
> 
> Maybe add a FAQ also explaining: why does the number of crashes per active
> daily user spike shortly after a release? Why do all 3 curves move together
> upwards/downwards on certain days?

It's not named clearly labeled a FAQ but I put a section about this in https://wiki.mozilla.org/Socorro/SocorroUI/Data_Assumptions_and_Gotchas#Crashes_per_ADU_spikes_shortly_after_a_release 

I don't believe we link that from the UI but we should consider doing so (perhaps renaming it to FAQ as well).

I think the remaining action for this bug is to correct labeling of the axes of the graph on the main home page dashboard, and to take any open questions to irc or the newsgroup (which should result either in more bugs filed or added to the FAQ).

Please let me know if I'm missing anything.

Comment 8

[Laura Thomson :laura]

+1 for correct axes labeling.

Comment 9

[Brandon Savage [:brandon]]

Attachment: Adding improved labeling

Comment 10

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: Post-fix screenshot

Comment 11

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED on:

https://crash-stats-dev.allizom.org/daily?form_selection=by_version&p=Firefox&v[]=5.0b1&throttle[]=100.00&v[]=6.0a1&throttle[]=100.00&v[]=4.2a1pre&throttle[]=100.00&v[]=4.0&throttle[]=10.00&hang_type=any&os[]=Windows&os[]=Mac&os[]=Linux&date_start=2011-05-09&date_end=2011-05-23&submit=Generate#

Comment 12

[Robert Kaiser]

(In reply to comment #10)
> Created attachment 534945 [details]
> Post-fix screenshot

Nice - same on the front page graph?

Comment 13

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #12)
> (In reply to comment #10)
> > Created attachment 534945 [details]
> > Post-fix screenshot
> 
> Nice - same on the front page graph?

Crud; missed that -- reopening, sorry.

Comment 14

[Brandon Savage [:brandon]]

Fixed in Subversion revision 3175.

Comment 15

[Stephen Donner [:stephend] Not actively reading bugmail]

Looks good on https://crash-stats-dev.allizom.org/products/Thunderbird/versions/3.1.9, https://crash-stats-dev.allizom.org/products/Firefox, etc.

Kairo, mind giving it an additional once-over?

Comment 16

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: Post-fix screenshot (for the product version's homepage)

Comment 17

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED per comment 15, comment 16.

Comment 18

[Benoit Jacob [:bjacob] (mostly away)]

Thanks!