make topcrasher reports work for each beta and release version

VERIFIED FIXED in 2.2

Status

Socorro
General
P1
blocker
VERIFIED FIXED
9 years ago
7 years ago

People

(Reporter: chris hofmann, Assigned: rhelmer)

Tracking

Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [project killwolf])

(Reporter)

Description

9 years ago
several days ago we make the first release candidate builds for firefox 3.6 ( build id# 2010 01 05 212446, for windows, and similar for mac and linux )  and we started using reports like this to study crashes

http://crash-stats.mozilla.com/topcrasher/byversion/Firefox/3.6/3

that worked great, but now we have a new set of respin builds with the same product identifier and version.

for tracking multiple release candidates, which we frequently do the the end-game shipping cycles,  we also need the ability to filter on just the set of new build ids.   the query might look something like

http://crash-stats.mozilla.com/topcrasher/byversion/Firefox/3.6/3?build_id=20100115144158,mac_build_id,linux_build_id

Comment 1

8 years ago
Priority?

Updated

8 years ago
Severity: normal → enhancement
(Reporter)

Comment 2

8 years ago
the next big release  ( firefox.next ) where we have multiple RC's sounds like its scheduled this fall at earliest.  that's mostly when this will be needed.

Comment 3

8 years ago
OK, scheduling for 2.0.
Target Milestone: --- → 2.0
(Reporter)

Comment 4

8 years ago
actually we ran into this with 3.6.4, and the addition of OOPP where better filtering of the standard reports are needed as we get down the the end of a release cycle.   one thing that might help is to put the 'advanced filtering options'  on the main top crash page.

So what is on
http://crash-stats.mozilla.com/topcrasher/byversion/Firefox/3.6.4

would also be shown with the ability to apply filter rules

 Product = Firefox
 Version = 3.6.4
 Build ID = 20100527000000 (actually want this to show that build id or greater)
 Report Process = browser
 Report Type = crash
Target Milestone: 2.0 → 1.7

Comment 5

8 years ago
1.7 is frozen, but we can take this for 1.8 which will be a super short cycle.
Target Milestone: 1.7 → 1.8

Comment 6

8 years ago
Ryan, can you take a look at this one?
Assignee: nobody → ryan
Target Milestone: 1.8 → 1.9
Assignee: ryan → nobody

Comment 7

8 years ago
We'll probably run into this soon, when FF4 RCs come around, just as a heads-up...

Comment 8

8 years ago
Thanks for surfacing this, it was way off the radar.  

I don't know if we can get it done in 1.7.7, but that's the only release timeline that would work for Firefox RCs.

I *think* this might be fastest implemented as a new view, but I want Ryan's input as he's done recent work on TCBS and I'm not that familiar with the internals.  Ryan, your thoughts?
Target Milestone: 1.9 → 1.7.7

Comment 9

8 years ago
As a note, there seem to be other requests around for being able to look at least at topcrashers by build ID, e.g. bug 555831 - I guess that would help analysis for nightlies, multiple builds per release and RCs all at once.
(Reporter)

Comment 10

8 years ago
I think we might be able to construct the queries to get the data needed with the existing features in socorro that allow specification of build id's.  the trick to fixing this bug would then just be to hook up a page to surface those queries in something like the "nightly build page" https://crash-stats.mozilla.com/products/Firefox/builds  but constructed from the list of release candidates on the ftp site.
Summary: make reports work for release candidate build tracking. → make topcrasher reports work for release candidate build tracking.
This isn't something that we can do with a quick fix to TCBS.  This is something we should definitely do, but we need to do it right, e.g. not a temporary hack.  We should add the ability to filter TCBS results on build ID when we refactor / replace the existing code.

For now, the best thing to do is to use Advanced Search to query against reports matching a specific Product / Version / Build ID.  Results are sorted by top crashing signature by default.  Some information is missing from Advanced Search results, like Trends and # of versions, but it should be enough to suffice through the RC period.

Updated

8 years ago
Target Milestone: 1.7.7 → 1.7.8
(Reporter)

Comment 12

7 years ago
if there is anything we can do quickly here we could use it since Firefox 4 RC 2 will be spinning this afternoon.  

I'll also do a set of top crash reports that just list RC2 and put them on people.m.o for the next few days.
(In reply to comment #12)
> if there is anything we can do quickly here we could use it since Firefox 4 RC
> 2 will be spinning this afternoon.  
> 
There's not, unfortunately.  It's non trivial to fix.
(Reporter)

Comment 14

7 years ago
I have reports running ready to pick up crashes from only 4.0 builds after 
2011 03 16 and produce a daily top crash list.

See:
http://people.mozilla.org/~chofmann/crash-stats/20110318/top-4.0RC2.html

Also have a rank comparison between RC2 and RC1 to help spot any unexpected crash regressions in RC2.

http://people.mozilla.org/~chofmann/crash-stats/20110318/compare-rank-4.0RC2-4.0.txt

the reports for RC2 still have too little volume and are too noisey to be useful but that should change tomorrow and the next day as RC2 builds get pushed to the beta channel.

Updated

7 years ago
Assignee: nobody → laura
This issue is going to get bad very quickly with our new rapid release plan.
Betas will all have the same version and when a release is made, the version still wont change.  The only way to tell an old beta crash from a release crash would be by the build id or by the channel.

I'd recommend we evaluate how quickly we can get both build id and channel into the breakpad submitted data or else people will likely be confused when we get farther along into the beta cycle

Comment 16

7 years ago
(In reply to comment #15)
> I'd recommend we evaluate how quickly we can get both build id and channel into
> the breakpad submitted data

AFAIK, build ID is in the submitted data, at least crash reports like e.g. https://crash-stats.mozilla.com/report/index/2217ab32-732f-400f-8ae8-a793c2110425 list it - we just don't have a much we are actually doing with it on the Socorro side. The channel is a different topic as it's really not submitted, and bug 649419 is filed on that. You might want to emphasize there why you think we need to get it.

Updated

7 years ago
Summary: make topcrasher reports work for release candidate build tracking. → make topcrasher reports work for multiple build IDs with same version

Updated

7 years ago
Blocks: 652880
Lars and I have a plan to fix this and related bugs: will take a week or two to implement though.
Whiteboard: [project killwolf]
Target Milestone: 1.7.8 → 2.0

Updated

7 years ago
Assignee: laura → lars
Target Milestone: 2.0 → 2.1
Depends on: 667101
Part of the 2.2 focus.
Target Milestone: 2.1 → 2.2
Tag team this with lonnen, please.
Severity: enhancement → blocker
Priority: -- → P1
(Assignee)

Updated

7 years ago
Assignee: lars → rhelmer
This is basically just using the new aggregation, as discussed, for betas and releases in TCBS.

We will NOT work on TCBU or TCBD in 2.2.  We will add those reports with the new aggregation at a later date.
No longer depends on: 667101
Summary: make topcrasher reports work for multiple build IDs with same version → make topcrasher reports work for each beta and release version
(Assignee)

Updated

7 years ago
Depends on: 676327
(Assignee)

Updated

7 years ago
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
QA verified, using the new aggregation method for betas and relases in TCBS
Status: RESOLVED → VERIFIED
Component: Socorro → General
Product: Webtools → Socorro
You need to log in before you can comment on or make changes to this bug.