649432 - Make build channels available in Socorro

Status: VERIFIED FIXED

(Socorro :: General, task)

Description

[Robert Kaiser]

We ended up selecting a versioning scheme where beta and release can not be told apart by version, so we'll need to get the channel into crash-stats. Bug 649419 is filed on the Breakpad side, we'll need to care to get this information into Socorro once it's coming in from there.

Comment 1

[Robert Strong (they/them - no direct email)]

Specifically, the beta and release versions will be the same.
http://mozilla.github.com/process-releases/draft/development_specifics/

Comment 2

[Robert Kaiser]

Actually, gavin says in bug 649419 that this actually was already implemented in bug 526623 - if that's true, all that was and is still missing is the Socorro side!

Can someone verify that crash reports coming in actually do have a "ReleaseChannel" annotation that tells us the channel?

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to comment #2)
> Actually, gavin says in bug 649419 that this actually was already implemented
> in bug 526623 - if that's true, all that was and is still missing is the
> Socorro side!
> 
> Can someone verify that crash reports coming in actually do have a
> "ReleaseChannel" annotation that tells us the channel?

I chose some random crash IDs in production and found this in the raw JSON in HBase (which is where the data received by the breakpad HTTP POST is stored):

"ReleaseChannel": "release"
"ReleaseChannel": "beta"
etc.

I don't see this in the processed report.

Comment 4

[Robert Kaiser]

As a further note how we might want to use that code:

From bug 526623 comment #0:
> Doing so would allow us to easily process 100% of crash reports for users
> during the maintenance release beta periods we have. Otherwise, we'll have to
> unthrottle then re-throttle all the time.

Note that the latter is what is happening nowadays...

Also, we'd love to have that field in the CSVs so any experiments we do based on those can easily filter for the channel.

Comment 5

[Laura Thomson :laura]

Lars, can you please take the first part of this (getting it into the processed json) and then throw it back to me?

Comment 6

[[:jberkus] Josh Berkus]

So we'll need to add this column to the PG database as well, in some form.

Comment 7

[K Lars Lohn [:lars] [:klohn]]

I have checked in a change that makes the processors copy the "ReleaseChannel" from the raw json into the processed json.  This is checked into trunk.

Comment 8

[Laura Thomson :laura]

I'll get this into the UI.  Will put this into 1.7.7 branch for a .x release.

Comment 9

[Marcia Knous [:marcia]]

Laura: A question came up in the beta meeting - after this change is made will we able to capture crashes that have already happened with Firefox 5 Beta 1? 

I have already created a 5.0b1 entry in the Socorro UI.

Comment 10

[(not currently active) Ted Mielczarek]

Those crashes will never show up as 5.0b1, since their version is being sent as 5.0. If we need to display them differently somehow, we'll need to sort out the details first and then file another bug to implement that. This sounds like something that ought to be hashed out at a planning meeting first.

Comment 11

[Sheila Mooney]

I say we focus on getting all future crash reports in the right place then we can talk about the old data. For now it's probably just QA so in my opinion it would not be catastrophic if we didn't get these reports.

Laura, if we put this into 1.7.7 branch, when would it go out?

Comment 12

[Laura Thomson :laura]

The change we're talking about here is just adding the ReleaseChannel information to the UI for an individual crash, but I see the need to get the data sliced by channel.  Adding this one piece of data is something we can ship as a dot release, probably even this week.
  
Sheila or Marcia: can you specify exactly what you want?  Hard to know how long the other stuff is going to take when I'm not sure of the scope yet.

Comment 13

[Marcia Knous [:marcia]]

Laura: Right now our most urgent need is to be able to search in the UI for 5.0b1 crash reports (because the version is being sent as 5.0). Would you like me to file a new bug for that?

Comment 14

[Laura Thomson :laura]

Marcia: yes please!

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

Marcia, you could do an advanced search and filter on the buildID of 20110427143820. For the last few releases all the platforms have had the same buildID.

Hopefully there is a bug elsewhere on this, but what Ted says in comment #10 is absolutely right. With several sets of builds reporting with version 5.0, we have to use update channel *and* buildID to tell between them. ie if you want to know how crashy each set of builds on the beta channel is, compared to the previous ones.

Comment 16

[Laura Thomson :laura]

Attachment: Pass the ReleaseChannel through and display it in the webapp

This just passes it through to an individual crash report display in the webapp.
Depends on lars' commits in trunk, r3127 and r3128.

Comment 17

[Laura Thomson :laura]

Also, I had a phone conversation with Marcia and Sheila yesterday about this, outcomes as follows:
- QA will use the advanced search as a temporary measure
- I needed to check if ADUs were available by buildid and apaprently they are. I don't think we have them this way in Socorro (yet).
- Longer term, we'll need to slice by version + buildid  for betas and final releases, and version alone for Aurora and Nightlies.
- The betas and final reports may be easy to pull out with ElasticSearch.

We'll merge this bug to 1.7.7 branch, tag it as 1.7.7.1 and push it out ASAP (today preferably).

Comment 18

[Laura Thomson :laura]

That should be 1.7.7.2, btw.

Committed to trunk:
Sending        webapp-php/application/libraries/CrashReportDump.php
Sending        webapp-php/application/views/report/index.php
Transmitting file data ..
Committed revision 3137.

Lars, can you merge your changesets above to the release branch please?
http://socorro.googlecode.com/svn/branches/1.7.7/

Comment 19

[Laura Thomson :laura]

Make that branches/releases/1.7.7.

Comment 20

[K Lars Lohn [:lars] [:klohn]]

my changes are now checked into branches/releases/1.7.7
Committed revision 3138.

Comment 21

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #17)
> - Longer term, we'll need to slice by version + buildid  for betas and final
> releases, and version alone for Aurora and Nightlies.

FWIW, I think it would be useful to slice aurora & nightly by buildid too, so we can easily say <this> nightly is really crashy and this <this later one> is better. Feel free to point me to a more appropriate place to discuss this :-)

Comment 22

[Laura Thomson :laura]

Merged changes in revisions 3138, 3139, 3140.

To test, we'll need to submit a few crashes to staging for processing, and then
put the OOIDS in here so QA can look.  Jabba/Rhelmer, can you push some crashes
for us?

Nick: we should spin a tracker bug off for other changes.

Comment 23

[Matt Brandt [:mbrandt]]

QA verified. Both Aurora and Beta channels are being marked as such in the crash reports.

Aurora Channel
https://crash-stats.allizom.org/report/index/3cf4e0b3-9cc1-42e9-b7b1-ab2972110504
Beta Channel:
https://crash-stats.allizom.org/report/index/2910c3a5-3fd9-4201-8a41-5457b2110504

Comment 24

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to comment #23)
> QA verified. Both Aurora and Beta channels are being marked as such in the
> crash reports.
> 
> Aurora Channel
> https://crash-stats.allizom.org/report/index/3cf4e0b3-9cc1-42e9-b7b1-ab2972110504
> Beta Channel:
> https://crash-stats.allizom.org/report/index/2910c3a5-3fd9-4201-8a41-5457b2110504

...and "release": https://crash-stats.allizom.org/report/index/80759c44-edeb-48f7-91f0-206892110504

Comment 25

[Robert Helmer [:rhelmer]]

Thanks mbrandt/stephend!

Since we're doing Hbase maintenance this evening, we are going to push this live tomorrow morning.

Comment 26

[Matt Brandt [:mbrandt]]

Thanks for the great assist :) What time is this being pushed? I want to be online.