657400 - Make it possible to get crashes per user for each beta and release version

Status: VERIFIED FIXED

(Socorro :: General, task, P1)

Description

[Robert Kaiser]

For analyzing the betas for Firefox 5 and higher, we'll want to have crashes per user graphs for comparing different betas, which we can only tell apart by Build ID + channel.

Comment 1

[Robert Kaiser]

Probably will need metrics to be involved, I guess.

Comment 2

[chris hofmann]

we will probably need a pretty tight spec on this before we implement.

I think what this means is that we need the ability to specify a time frame for each of the releases, and probably to isolate on a specfic build id, or set of builds as well.


right now we specify 

 list of versions - list of throttle values
 
 date span for all above

So for the comparision chart I think we need

 list of versions -build date - list of throttle values - begin date - end date 


where build date might be:
  a single date, 
  list of comma separated dates, 
  or build date >= value

Comment 3

[Robert Kaiser]

Right now (with decreasing priority), it would be helpful if we could

1) For beta and release channels, have separate crashes/ADU even if the version number is the same.
I.e. right now, if we had "5.0 Beta" with unthrottled crashes/ADU and as a separate thing "5.0 Release" with throttled crashes/ADU, we would have two meaningful numbers while now we have one bogus number.

2) For Nightly and Aurora, if we had (crashes for all build IDs of a day) / (sum of all daily ADUs for those build IDs), this would give us a more meaningful graph than what we have right now.

3) For Betas, comparing numbers like in 2 for different beta builds would be very helpful as well.

Comment 4

[Laura Thomson :laura]

Assigned to lars, but this please tag team it with lonnen.

Comment 5

[[:jberkus] Josh Berkus]

Kairo,

The problem with this is that products other than Firefox do not have consistent buildIDs across all platforms.  So it's unclear on what build_id we would use.

Comment 6

[Laura Thomson :laura]

To change the summary: this is just what we've been talking about: changing the aggregation for crashes per user for betas and releases.

In actuality, we're implementing this alongside the existing crashes/user as that's easier.

Comment 7

[chris hofmann]

(In reply to comment #5)
> Kairo,
> 
> The problem with this is that products other than Firefox do not have
> consistent buildIDs across all platforms.  So it's unclear on what build_id
> we would use.

Its not clear to me why we would need to hold up this high priority Firefox need for other products that may not have consistent build ids.  The default should be to match all build ids like we do now.  other products could continue to work just as they do now.   

otherwise we could match on a list of user provided build ids whatever their format.

Comment 8

[Laura Thomson :laura]

We're not holding it up for other products, or for anything else: pretty much the entire team is working on this (and the related bug for topcrashers per beta) as a top priority, and will be doing so until it's shipped.

Comment 9

[Robert Kaiser]

(In reply to comment #5)
> Kairo,
> 
> The problem with this is that products other than Firefox do not have
> consistent buildIDs across all platforms.  So it's unclear on what build_id
> we would use.

Sigh. As Laura already pointed out, we need it per-what-we-consider-to-be-one-build, as comment #0 already pints out, the target is to be able to tell apart different Firefox betas, and as Laura also pointed out, we don't want to hold any Firefox progress up for other products. I hope that solves this question once and for all.

Comment 10

[Matt Brandt [:mbrandt]]

Attachment: adu data

QA verified. ADU metrics include build channel