Closed Bug 869905 Opened 11 years ago Closed 11 years ago

Make Socorro work with the new B2G release channels

Categories

(Socorro :: General, task)

Product:
Socorro
Component:
General
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: kairo, Assigned: rhelmer)

References

Details

It looks like bug 861426 and bug 862526 are doing major surgery with release channels for B2G and that could mess up any assumptions Socorro has about channels. Somehow those things needs to work together.
It looks like catlee is changing things around in a way that we continue to have the "traditional" release channels set there, so we probably won't need any real work here.
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #1)
> It looks like catlee is changing things around in a way that we continue to
> have the "traditional" release channels set there, so we probably won't need
> any real work here.

I had a look, and it seems like we don't need to do anything. Can we close this then, KaiRo?
(In reply to Selena Deckelmann :selenamarie :selena from comment #2)
> I had a look, and it seems like we don't need to do anything. Can we close
> this then, KaiRo?

Well, catlee hasn't changed the releng systems "back" to the normal channels yet, I'd like to keep this open until he has done that and we can verify that what we did actually does not need work on our side.
catlee: when are they changing?
Flags: needinfo?(catlee)
catlee, ping?
ping?
I thought that the current channels worked for socorro -- is that incorrect?
Flags: needinfo?(laura)
Flags: needinfo?(kairo)
I am not sure, to be honest.  Kairo, what is the actual ask here? The channels we have now are the ones we are keeping. What do you expect to see? We can't do graphs and topcrashers without ADU, so...
Flags: needinfo?(laura)
RelEng still set the channels not to "nightly" but to e.g. "hamachi/1.2.0/nightly" (see e.g. bp-94850f3d-cf16-4ef8-ae89-f7b092130916) which is something we do not support in Socorro and catlee has said that they will switch away from again and back to the standard "nightly" etc. - but this still has not happened, as you can see in the mentioned crash report, for example.
Flags: needinfo?(kairo)
(In reply to Laura Thomson :laura from comment #8)
> We can't do graphs and topcrashers without ADU, so...

We can't do graphs, that's right, but we should see to being able to do topcrashers as they don't need ADI numbers by themselves.
from irc w/laura just now:

1) aki thinks that https://bugzilla.mozilla.org/show_bug.cgi?id=861426 is already (long ago) fixed, so its now closed. If something is missing, please reopen with details.
 
2) For some months now now, b2g builds have been using "device-name/version/{nightly,beta}" format. For example, http://update.boot2gecko.org/unagi/1.1.0/nightly/update.xml is for nightly builds for unagi devices for v1.1.0... and we hope is being counted by crash-stats as a nightly build.

3) We think the format is not changing, but there was some confusion by the recent v1.2.0 branch, so later today, we'll start making updates on http://update.boot2gecko.org/unagi/1.2.0/nightly/update.xml. Note: that is same format, but using 1.2.0 instead of 1.1.0. 


As updates have been in this format for a while, maybe we are all ok here, and there is some confusion on whether anything left to do here or not; laura and rhelmer investigating and will get back to us.
Ok.

I'm not the expert here, but, aiui from vague recollections:

* catlee had to solve the multiple device/branch channel issue, which beta+nightly couldn't support
* catlee tried renaming the channel DEVICE-CHANNEL or something, and that broke socorro, which expects beta or nightly or release
* catlee then made the update path DEVICE/VERSION/CHANNEL, where channel is beta or nightly

* aki assumed this meant that socorro now saw beta or nightly as the channel, and all was good in the world

* many moons later, joduinn points out this bug to aki and now we need to figure out if the above assumption is true, or if things are still broken.

We can't move back to a single 'nightly' or 'beta' channel because we have to support multiple devices across multiple branches.  Possible solutions I see off the top of my head:

* make socorro smarter
* rewrite b2g update system + server for all branches and devices to accomodate socorro
* restrict all of qa/eng/community to a single device type
Mapping channels on the Socorro side should not be big deal, as long as we agree on a scheme. However - on https://ftp.mozilla.org/pub/mozilla.org/b2g/manifests/nightly/ I only see release channels I'd expect (nightly, beta, some are on default). Looking at raw incoming reports for this month so far:

 count |  release_channel   
-------+--------------------
 52520 | nightly
     2 | unagi/1.1.0/beta
    20 | nightly-crespo
     1 | leo/1.1.0/nightly
     1 | beta

We could easily parse the channel and pull out the data we're interested in, and correlate it to the data we get from FTP.

There are some other issues that I think we need to deal with right now though, for instance since Socorro uses the gecko version number exposed by e.g. https://ftp.mozilla.org/pub/mozilla.org/b2g/manifests/nightly/1.1.0/latest/socorro_hamachi.json it's making the assumption that this expires every 6 weeks. So, it looks like we are picking data up but it's expired so not showing up in the UI anymore:

 sunset_date | build_type | version_string 
-------------+------------+----------------
 2013-10-30  | nightly    | 26.0a1
 2013-09-25  | nightly    | 25.0a1
 2013-08-23  | nightly    | 24.0a1
 2013-06-20  | beta       | 18.0b
 2013-06-20  | beta       | 18.0b1

Since B2G 1.0.x and 1.1.x are based on gecko 18.0, they are by far the highest crash volume (here is count of raw crashes since Sept 1st):

 version | count 
---------+-------
 16.0a1  |     1
 17.0a1  |    17
 18.0    | 52547
 24.0a1  |     2
 25.0a1  |     5
 26.0a1  |  1269
 27.0a1  |     6

Socorro isn't attempting to do reporting on any of these crashes, and they aren't showing up in the pull down menus in the UI.

Short-term, we can set the sunset date for 18.0 far into the future, and start working on parsing the new release channels. However, the implication here is that we're lumping all 1.0.x and 1.1.x releases under "18.0", it would probably make sense for Socorro to understand B2G release versions and expose those instead.
Assignee: nobody → rhelmer
Kairo, besides pointing out comment 13 I want to make sure we I understand what you are expecting out of this bug - currently the only Socorro features that work (as far as I am aware) for B2G are search and pulling up individual crash reports (aka /report/index/CrashID).

In principle I think reports that don't have a hard requirement on ADI like TCBS could be made to work, which is out of scope for but would be dependent on this bug getting the channel mapping described in comment 13 right (should we just file that now?)
Flags: needinfo?(kairo)
I went ahead and updated the sunset date for 18.0b*:

 sunset_date | version_string 
-------------+----------------
 2014-01-01  | 18.0b1
 2014-01-01  | 18.0b

This should make it show up in the UI at least, so it's selectable in search.
(In reply to Aki Sasaki [:aki] from comment #12)
> * catlee then made the update path DEVICE/VERSION/CHANNEL, where channel is
> beta or nightly

We (at least laura and me) had a Vidyo meeting with catlee at that point, where he promised to revert this and solve the updates in a different (and IMHO more correct) way because this would take quite some work for Socorro to map things correctly and the channels used now are probably also breaking some assumptions in the source code where at some points the start of the channel name is being used to determine which functionality to activate.
(Also, the slashes in the channel name make up for a security hazard where channels are used in URLs to data.)

Rob, the version handling in Socorro is also a bug, but let's deal with that in a different place, we have multiple hazards there. One of them is that we need to do something to version handling, another is that FTP scraping is only giving us a fraction of the existing B2G build as the most important builds are those from partners/manufacturers which we get no build data for at all, not on FTP and not elsewhere other than in crash reports themselves.


(In reply to Robert Helmer [:rhelmer] from comment #14)
> Kairo, besides pointing out comment 13 I want to make sure we I understand
> what you are expecting out of this bug

What I expect out of this bug is ideally catlee doing what he promised to do and revert channels for the testing-only build done by releng to simple values not including slashes. If releng decides to not do that, I expect the Socorro team to view it as a top-priority item to make Socorro deal with the hackish channel names releng is setting there.
Flags: needinfo?(kairo)
(In reply to Robert Helmer [:rhelmer] from comment #15)
> I went ahead and updated the sunset date for 18.0b*:
> 
>  sunset_date | version_string 
> -------------+----------------
>  2014-01-01  | 18.0b1
>  2014-01-01  | 18.0b
> 
> This should make it show up in the UI at least, so it's selectable in search.

Thanks, but that's bug 915257, and that date is not what's being requested there.
I filed bug 918068 on fixing the channel names on the releng side, if that is fixed, this bug here is not needed.
Depends on: 918068
We agreed to close this in favor of more specific bugs: bug 918068 and  bug 917976.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → INCOMPLETE
Flags: needinfo?(catlee)
You need to log in before you can comment on or make changes to this bug.