Last Comment Bug 641484 - Document/clarify what 'Crashes Per Active Daily User' means
: Document/clarify what 'Crashes Per Active Daily User' means
Status: VERIFIED FIXED
:
Product: Socorro
Classification: Server Software
Component: General (show other bugs)
: Trunk
: All All
: -- normal (vote)
: 1.7.8
Assigned To: Brandon Savage [:brandon]
: socorro
Mentors:
https://crash-stats-dev.allizom.org/d...
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-03-14 06:51 PDT by Benoit Jacob [:bjacob] (mostly away)
Modified: 2011-12-28 10:40 PST (History)
6 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Adding some sort of description (901 bytes, patch)
2011-05-03 07:17 PDT, Brandon Savage [:brandon]
rhelmer: review+
Details | Diff | Review
Adding improved labeling (1.29 KB, patch)
2011-05-18 14:10 PDT, Brandon Savage [:brandon]
rhelmer: review+
Details | Diff | Review
Post-fix screenshot (625.49 KB, image/png)
2011-05-24 17:17 PDT, Stephen Donner [:stephend]
no flags Details
Post-fix screenshot (for the product version's homepage) (202.36 KB, image/png)
2011-05-25 14:05 PDT, Stephen Donner [:stephend]
no flags Details

Description Benoit Jacob [:bjacob] (mostly away) 2011-03-14 06:51:20 PDT
The graphs on the crash-stats home page are very hard to make sense of. What does the y-axis unit mean? How is 'Active Daily User' defined? Has every user really crashed twice in average? Does that include plugin crashes (I think it shouldn't)?

Some people say it's not per user but per-100-users, is that correct?

Maybe add a FAQ also explaining: why does the number of crashes per active daily user spike shortly after a release? Why do all 3 curves move together upwards/downwards on certain days?
Comment 1 Robert Kaiser (not working on stability any more) 2011-03-14 08:32:11 PDT
I guess linking to a wiki page that describes "common trend patterns" or so would be a good idea for the latter part.

The others should added directly on the page as some sort of description to give people who don't know the intrincics knowledge what this really is about.
Comment 2 Brandon Savage [:brandon] 2011-04-22 11:30:20 PDT
Moving onto the 1.7.8 milestone.
Comment 3 [:jberkus] Josh Berkus 2011-04-22 18:10:57 PDT
I've documented this on the assumptions etc. wiki page, but there really needs to be something on the ADU page itself.
Comment 4 Brandon Savage [:brandon] 2011-05-03 07:17:10 PDT
Created attachment 529714 [details] [diff] [review]
Adding some sort of description

I used the text provided by jberkus on the assumptions page for the definition. It's placed in the search box so that users can understand the term, and effectively make decisions based on the definition. If you have a better place to put it, or improved wording, please r- and let me know what your thoughts are.
Comment 5 Brandon Savage [:brandon] 2011-05-04 08:46:03 PDT
Fixed in revision 3134.
Comment 6 Benoit Jacob [:bjacob] (mostly away) 2011-05-04 08:54:38 PDT
Hey! The patch here only adds a description that is certainly useful, but still the labeling of the axes of the graph shown on the crash-stats main page is not updated, and according to the clarification from your patch, it's wrong! (It should say "crashes per 100 ADUs" instead of "crashes per ADU" iiuc).
Comment 7 Robert Helmer [:rhelmer] 2011-05-04 14:42:37 PDT
(In reply to comment #6)
> Hey! The patch here only adds a description that is certainly useful, but still
> the labeling of the axes of the graph shown on the crash-stats main page is not
> updated, and according to the clarification from your patch, it's wrong! (It
> should say "crashes per 100 ADUs" instead of "crashes per ADU" iiuc).

Reopening to cover this. We also have been discussing some open questions from comment 0:

(In reply to comment #0)
> The graphs on the crash-stats home page are very hard to make sense of. What
> does the y-axis unit mean? How is 'Active Daily User' defined? Has every user
> really crashed twice in average? Does that include plugin crashes (I think it
> shouldn't)?


It'd probably be clearer to track the plugin crash questions as a separate bug. If you want to drop by IRC (#breakpad) or post to mozilla.tools.socorro we can help figure out if this is a bug or more something for the FAQ.


> Some people say it's not per user but per-100-users, is that correct?
> 
> Maybe add a FAQ also explaining: why does the number of crashes per active
> daily user spike shortly after a release? Why do all 3 curves move together
> upwards/downwards on certain days?

It's not named clearly labeled a FAQ but I put a section about this in https://wiki.mozilla.org/Socorro/SocorroUI/Data_Assumptions_and_Gotchas#Crashes_per_ADU_spikes_shortly_after_a_release 

I don't believe we link that from the UI but we should consider doing so (perhaps renaming it to FAQ as well).

I think the remaining action for this bug is to correct labeling of the axes of the graph on the main home page dashboard, and to take any open questions to irc or the newsgroup (which should result either in more bugs filed or added to the FAQ).

Please let me know if I'm missing anything.
Comment 8 Laura Thomson :laura 2011-05-05 07:25:56 PDT
+1 for correct axes labeling.
Comment 9 Brandon Savage [:brandon] 2011-05-18 14:10:45 PDT
Created attachment 533415 [details] [diff] [review]
Adding improved labeling
Comment 10 Stephen Donner [:stephend] 2011-05-24 17:17:57 PDT
Created attachment 534945 [details]
Post-fix screenshot
Comment 12 Robert Kaiser (not working on stability any more) 2011-05-25 06:38:28 PDT
(In reply to comment #10)
> Created attachment 534945 [details]
> Post-fix screenshot

Nice - same on the front page graph?
Comment 13 Stephen Donner [:stephend] 2011-05-25 09:32:32 PDT
(In reply to comment #12)
> (In reply to comment #10)
> > Created attachment 534945 [details]
> > Post-fix screenshot
> 
> Nice - same on the front page graph?

Crud; missed that -- reopening, sorry.
Comment 14 Brandon Savage [:brandon] 2011-05-25 13:38:15 PDT
Fixed in Subversion revision 3175.
Comment 15 Stephen Donner [:stephend] 2011-05-25 13:58:13 PDT
Looks good on https://crash-stats-dev.allizom.org/products/Thunderbird/versions/3.1.9, https://crash-stats-dev.allizom.org/products/Firefox, etc.

Kairo, mind giving it an additional once-over?
Comment 16 Stephen Donner [:stephend] 2011-05-25 14:05:48 PDT
Created attachment 535177 [details]
Post-fix screenshot (for the product version's homepage)
Comment 17 Stephen Donner [:stephend] 2011-05-25 14:06:13 PDT
Verified FIXED per comment 15, comment 16.
Comment 18 Benoit Jacob [:bjacob] (mostly away) 2011-05-25 14:10:51 PDT
Thanks!

Note You need to log in before you can comment on or make changes to this bug.