821989 - Turn on daily/rapid beta support for B2G

Status: VERIFIED FIXED

(Socorro :: General, task)

Description

[Robert Kaiser]

As B2G is using the "beta" channel now and gets gets builds (with roughly one weekly being promoted to the general testing public, I think), this should be a good testing ground for Socorro's "rapid beta" support and we should turn that on for B2G 18.

That said, the FTP scraping can be tricky as AFAIK we are building off the mozilla-b2g18 repository and not mozilla-beta, and we might be using the nightly build mechanism and not the usual beta/release harness.

John, where are we putting the build metadata of unagi builds that have the beta channel set, on in what format? Can you point to an example? I hope that's in a place that Socorro can read easily enough so rhelmer can hook that scraping up.

Though, if I think about it, scraping for desktop builds may be enough if the "rapid release" stuff doesn't need build IDs to match 100%.

Comment 1

[Laura Thomson :laura]

When is this needed/how high a priority?

Comment 2

[Robert Kaiser]

(In reply to Laura Thomson :laura from comment #1)
> When is this needed/how high a priority?

Well, we want it as soon after the filing of this bug as possible as we've been on the beta channel since then, and we have the code freeze set for Jan 15 so having data before then would be good. Well, strike that, don't want to be overly sarcastic, but needless to say that we would have needed it for some time already. Still will do good when done now or in the very near future.

We won't get any useful reports at all on the builds used by our unagi test drivers without this bug fixed.

I think rhelmer hasn't touched this mostly because I asked joduinn a question in comment #0 and there hasn't been any answer. Now I just guessed at what Rob might need when I asked this question, but if that's what we need to get this done, we should needinfo? or ping John.

Comment 3

[Robert Helmer [:rhelmer]]

OK investigated - looks like:

1) enabling this on the Socorro side should be simple - I think all we need to do is set the rapid_beta_version column on the products table, and we should be able to backfill if that's helpful

2) we only have releases_raw entries for Nightly and Aurora for B2G, so if metadata is on FTP already we're not finding it (will look into this next)

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #3)
> 2) we only have releases_raw entries for Nightly and Aurora for B2G, so if
> metadata is on FTP already we're not finding it (will look into this next)

OK this bit is going to need some work - device builds are uploaded to a private area, and use XML files not the old .txt metadata files. I'll file a bug blocking this one.

Comment 5

[Robert Helmer [:rhelmer]]

bug 832083 is landed but still needs testing, so bumping this (the trivial part) to next week.

Comment 6

[Robert Helmer [:rhelmer]]

Working on landing this today.

Comment 7

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #3)
> OK investigated - looks like:
> 
> 1) enabling this on the Socorro side should be simple - I think all we need
> to do is set the rapid_beta_version column on the products table, and we
> should be able to backfill if that's helpful


OK rapid_beta_release_version is 17.0 in products table for b2g so this should be fine.


> 2) we only have releases_raw entries for Nightly and Aurora for B2G, so if
> metadata is on FTP already we're not finding it (will look into this next)

This should have been fixed in dependent bug 832083 and bug 832074

Comment 8

[Robert Helmer [:rhelmer]]

I *think* this should be working on stage (and prod if/when dependent bugs ship), let's test and figure it out if it's not.

Comment 9

[Lonnen :lonnen]

I don't see any commits for this. Is it in the right milestone?

Comment 10

[Robert Helmer [:rhelmer]]

(In reply to Chris Lonnen :lonnen from comment #9)
> I don't see any commits for this. Is it in the right milestone?

Sorry I wasn't very clear in comment 8, we've passed the rapid_beta_version defined in the DB so it *should* work.

Comment 11

[Matt Brandt [:mbrandt]]

Attachment: qa - verified on stage

QA verified on stage - what I see in the UI (/B2G/builds) matches my understanding of what we're seeking to capture. Kairo please reopen if I'm mistaken.

Comment 12

[Robert Kaiser]

(In reply to Matt Brandt [:mbrandt] from comment #11)
> QA verified on stage - what I see in the UI (/B2G/builds) matches my
> understanding of what we're seeking to capture. Kairo please reopen if I'm
> mistaken.

/B2G/builds only lists Nightly, but what counts for this is Beta...

We should pick up "18.0 Beta" and list topcrash reports, including those by build date, for that version when this bug is correctly resolved.

Comment 13

[Matt Brandt [:mbrandt]]

bumping to reopened per comment 12

Comment 14

[Lonnen :lonnen]

kick!

Comment 15

[Selena Deckelmann :selenamarie :selena]

KaiRo,

Could you give us a link to a Beta build that this should apply to? We've been sifting, but can't find one.

Thanks

Comment 16

[Robert Helmer [:rhelmer]]

(In reply to Selena Deckelmann :selenamarie :selena from comment #15)
> KaiRo,
> 
> Could you give us a link to a Beta build that this should apply to? We've
> been sifting, but can't find one.
> 
> Thanks

I would think this one would be beta but it is not:

I am having trouble finding *any* b2g manifests which use beta channel, e.g. 1.0.0 - http://ftp.mozilla.org/pub/mozilla.org/b2g/manifests/1.0.0/latest/socorro_unagi.json

Comment 17

[Robert Kaiser]

(In reply to Selena Deckelmann :selenamarie :selena from comment #15)
> Could you give us a link to a Beta build that this should apply to? We've
> been sifting, but can't find one.

I don't know where the JSON info for those actually ends up. I hope that catlee can point you to some build info JSON for those builds that end up on the B2G beta channel.

All I can tell you is that we have people out there crashing on the B2G 18 beta channel, e.g. bp-95afba3d-e174-4a76-8f14-2e2952130407

Comment 18

[Chris AtLee [:catlee]]

beta channel for b2g currently corresponds to "v1-train" (v1.1.0) builds. e.g.
http://ftp.mozilla.org/pub/mozilla.org/b2g/manifests/1.1.0/2013-04-08-23/socorro_unagi.json

Comment 19

[Laura Thomson :laura]

This was reopened: moving to an open release

Comment 20

[Robert Helmer [:rhelmer]]

18.0b1 and 18.0b show up for B2G now, so I think this is working. However crash volume is fairly low so we don't have many reports (topcrashers etc) yet.

You can see search results for beta crashes at least:
https://crash-stats.mozilla.com/query/query?product=B2G&version=B2G%3A18.0b1&range_value=1&range_unit=weeks&date=04%2F24%2F2013+17%3A25%3A00&query_search=signature&query_type=contains&query=&reason=&build_id=&process_type=any&hang_type=any&do_query=1

So I think the basic work is done here, if there are still problems or anything doesn't work as expected we should reopen.

Comment 21

[Matt Brandt [:mbrandt]]

Bumping to verified - This looks correct to me as well. Kairo does this pass your sniff test :)

Comment 22

[Robert Kaiser]

(In reply to Matt Brandt [:mbrandt] from comment #21)
> Bumping to verified - This looks correct to me as well. Kairo does this pass
> your sniff test :)

Well, Something smells not completely right with the data on prod from what I can see, but I'm not really sure what it is so far.

Comment 23

[Matt Brandt [:mbrandt]]

Reopening for investigation:

Fails the Kairo data sniff test (comment 22)

Comment 24

[Laura Thomson :laura]

When you figure out if something's wrong with it, reopen this or open a more specific bug.

Comment 25

[Matt Brandt [:mbrandt]]

I agree with comment 24 - closing this bug