Last Comment Bug 740829 - Nightly builds report for android doesn't have any data
: Nightly builds report for android doesn't have any data
Status: VERIFIED FIXED
:
Product: Socorro
Classification: Server Software
Component: Webapp (show other bugs)
: unspecified
: All All
: -- normal (vote)
: 6
Assigned To: Robert Helmer [:rhelmer]
:
Mentors:
Depends on:
Blocks: 740975
  Show dependency treegraph
 
Reported: 2012-03-30 08:53 PDT by Jeff Muizelaar [:jrmuizel]
Modified: 2012-04-23 18:13 PDT (History)
6 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
qa - verified (106.01 KB, image/png)
2012-04-18 09:14 PDT, Matt Brandt [:mbrandt]
no flags Details

Description Jeff Muizelaar [:jrmuizel] 2012-03-30 08:53:03 PDT
The following page is blank:
https://crash-stats.mozilla.com/products/FennecAndroid/versions/14.0a1/builds
Comment 1 Robert Helmer [:rhelmer] 2012-03-30 09:28:04 PDT
Josh can you take a look at this? From a quick look at update_nightly_builds() SP I don't see why it shouldn't be doing FennecAndroid.

I don't see any other reports having data on crash-stats other than Firefox, actually.
Comment 2 [:jberkus] Josh Berkus 2012-03-30 13:06:29 PDT
The problem is that the cron job for nightly_builds hasn't been run since March 22.  

So, assigning back to you.
Comment 3 Robert Helmer [:rhelmer] 2012-03-30 13:42:39 PDT
(In reply to [:jberkus] Josh Berkus from comment #2)
> The problem is that the cron job for nightly_builds hasn't been run since
> March 22.  
> 
> So, assigning back to you.

Yep, looks like we didn't enable the cronjob at the time that we switched over to using the matview. I think we should do a point release for this today.
Comment 4 Robert Helmer [:rhelmer] 2012-03-30 14:51:41 PDT
(In reply to Robert Helmer [:rhelmer] from comment #3)
> (In reply to [:jberkus] Josh Berkus from comment #2)
> > The problem is that the cron job for nightly_builds hasn't been run since
> > March 22.  
> > 
> > So, assigning back to you.
> 
> Yep, looks like we didn't enable the cronjob at the time that we switched
> over to using the matview. I think we should do a point release for this
> today.

OK pushed the fix to prod in bug 740975, back to you to backfill the matview :P
Comment 5 [:jberkus] Josh Berkus 2012-03-30 15:05:21 PDT
Backfilled.

Once cache times out, we can verify it.
Comment 6 Robert Kaiser (not working on stability any more) 2012-04-11 11:10:23 PDT
Umm, this still seems to not work, and I'm getting requests for this, as people want to see what crashes "are still happening" in current nightly builds.

Can we please look into this ASAP?
Comment 7 [:jberkus] Josh Berkus 2012-04-11 11:20:01 PDT
Has the cronjob been fixed?
Comment 8 Robert Helmer [:rhelmer] 2012-04-11 11:27:39 PDT
(In reply to [:jberkus] Josh Berkus from comment #7)
> Has the cronjob been fixed?

Yes, was pushed in bug 740975. I'll take a look.
Comment 9 Robert Helmer [:rhelmer] 2012-04-11 11:32:04 PDT
(In reply to Robert Helmer [:rhelmer] from comment #8)
> (In reply to [:jberkus] Josh Berkus from comment #7)
> > Has the cronjob been fixed?
> 
> Yes, was pushed in bug 740975. I'll take a look.

Yes job has been running, or at least logging successfully, since bug 740975 was pushed:

2012-04-11 02:19:14,629 INFO - Running update_nightly_builds

The SP it's calling is "update_nightly_builds", Josh can you make sure that's set up ok in the DB?
Comment 10 [:jberkus] Josh Berkus 2012-04-11 12:22:26 PDT
Ok, cron job confirmed working, and there's plenty of data in the matview for FennecAndroid 14.0a1.  So we should be seeing a report here.

So this is apparently a middleware or UI bug.  Needs to be reassigned.
Comment 11 Laura Thomson :laura 2012-04-11 12:26:34 PDT
Rob, can you puzzle this one out?
Comment 12 Robert Helmer [:rhelmer] 2012-04-11 17:29:18 PDT
(In reply to Laura Thomson :laura from comment #11)
> Rob, can you puzzle this one out?

Sure, taking a look now - Firefox, Thunderbird and SeaMonkey all seem to work find and be up to date, nothing for Fennec and FennecAndroid.
Comment 13 Robert Helmer [:rhelmer] 2012-04-11 18:33:00 PDT
OK here's the problem, in this middleware service:

https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/products_builds.py#L98

The SQL that gets generated is:
"""
/* socorro.external.postgresql.builds.Builds.builds */ 
SELECT  product_name as product,
         version,
         platform,
         build_id as buildid,
         build_type,
         beta_number,
         repository,
         build_date(build_id) as date 
FROM releases_raw 
WHERE product_name = E'FennecAndroid'
             
AND build_date(build_id) >=
    timestamp with time zone '2012-04-05'
AND repository IN ('mozilla-central', 'mozilla-1.9.2',
                   'comm-central', 'comm-1.9.2',
                   'comm-central-trunk')
ORDER BY build_date(build_id) DESC, product_name ASC, version ASC,
"""

However the product_name in releases_raw comes from FTP, so they look like:
seamonkey, firefox, mobile, thunderbird

Josh, should this be using product_info or some other table instead? I know we already have logic in the SPs to determine which "mobile" builds are Fennec versus FennecAndroid.
Comment 14 [:jberkus] Josh Berkus 2012-04-11 20:27:13 PDT
Robert,

Yes, it should be using product_info.
Comment 15 [:jberkus] Josh Berkus 2012-04-11 20:28:00 PDT
Or, if product_info doesn't have enough data, it should be using product_versions.
Comment 16 [:jberkus] Josh Berkus 2012-04-12 08:19:32 PDT
hmmm.

You know, let me write the query for this, since nobody on the development team has ever dealt with product_version_builds before.
Comment 17 [:jberkus] Josh Berkus 2012-04-12 14:26:25 PDT
Here's my suggested query:

SELECT DISTINCT product_versions.product_name,
  product_versions.version_string,
  product_version_builds.platform,
  product_version_builds.build_id,
  product_versions.build_type,
  product_versions.beta_number,
  build_date(product_version_builds.build_id) as build_date,
  product_versions.version_sort
FROM product_versions JOIN product_version_builds
  USING ( product_version_id )
WHERE product_name = E'FennecAndroid'
  AND build_date(build_id) >= DATE '$build_date'
ORDER BY build_date DESC, product_name ASC, version_sort ASC;

This loses the repository column, though, since we're not tracking that.
Comment 18 Robert Helmer [:rhelmer] 2012-04-12 16:19:48 PDT
(In reply to [:jberkus] Josh Berkus from comment #17)
> Here's my suggested query:
> 
> SELECT DISTINCT product_versions.product_name,
>   product_versions.version_string,
>   product_version_builds.platform,
>   product_version_builds.build_id,
>   product_versions.build_type,
>   product_versions.beta_number,
>   build_date(product_version_builds.build_id) as build_date,
>   product_versions.version_sort
> FROM product_versions JOIN product_version_builds
>   USING ( product_version_id )
> WHERE product_name = E'FennecAndroid'
>   AND build_date(build_id) >= DATE '$build_date'
> ORDER BY build_date DESC, product_name ASC, version_sort ASC;
> 
> This loses the repository column, though, since we're not tracking that.

Thanks, I think that's fine as we don't seem to be surfacing it in https://crash-stats.mozilla.com/products/Firefox/builds

I can work up a patch for this
Comment 19 Robert Helmer [:rhelmer] 2012-04-12 18:09:29 PDT
Adrian - r? https://github.com/mozilla/socorro/pull/500
Comment 20 Robert Helmer [:rhelmer] 2012-04-12 18:13:41 PDT
BTW I think we should take this for Socorro 6 (April 18) even though it missed freeze.
Comment 21 Robert Kaiser (not working on stability any more) 2012-04-13 07:07:39 PDT
(In reply to Robert Helmer [:rhelmer] from comment #20)
> BTW I think we should take this for Socorro 6 (April 18) even though it
> missed freeze.

Yes, please.
Comment 22 Robert Helmer [:rhelmer] 2012-04-13 14:36:34 PDT
OK so there are two potential issues with using the product_versions table for this, instead of the raw table:

1) it's only updated one per day (currently it is once-per-hour)
2) we can't restrict the repository without making some deeper changes which require more testing

I am assuming that #1 won't be a big problem (but throwing it out here now in case it is), but #2 makes it pretty unusable in it's current state.

So, since the product_builds service already hardcodes mozilla-specific config (repositories), I am temporarily adding hardcoding of Fennec*->mobile mappings as well.

For Socorro 7 (which will be released the week after 6), we can move all this stuff to the DB side and make it configurable so it won't break external installs.

This particular report is already broken for external installs, which is why I am OK adding more hardcoding for 6 :)
Comment 23 Robert Helmer [:rhelmer] 2012-04-13 14:37:03 PDT
Adrian, I have updated https://github.com/mozilla/socorro/pull/500
Comment 24 Robert Kaiser (not working on stability any more) 2012-04-13 15:55:41 PDT
(In reply to Robert Helmer [:rhelmer] from comment #22)
> 1) it's only updated one per day (currently it is once-per-hour)

Actually, that could be a problem, as we'd like to see the current nightly for a day on that report.

> 2) we can't restrict the repository without making some deeper changes which
> require more testing

I have no idea what we need this repository thing for in this report.


But as I guess you're doing it differently now anyhow, I guess both are not affecting thhis solution now anyhow. :)
Comment 25 Robert Helmer [:rhelmer] 2012-04-13 16:23:09 PDT
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #24)
> (In reply to Robert Helmer [:rhelmer] from comment #22)
> > 1) it's only updated one per day (currently it is once-per-hour)
> 
> Actually, that could be a problem, as we'd like to see the current nightly
> for a day on that report.


This is good to know, thanks. We might need to change strategy in this case, let's discuss in irc.


> > 2) we can't restrict the repository without making some deeper changes which
> > require more testing
> 
> I have no idea what we need this repository thing for in this report.

The FTP scraper picks up release metadata for everything posted, so we have been restricting it to just the major repositories (currently things like 'mozilla-central' are embedded in the source code, and should be configurable.)

Otherwise builds for all kinds of other repositories show up (dev branches like "oak", ESR, etc - basically all the different types of dir in http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/ - repository is really not the 100% correct name in many cases, it's more like a "conceptual branch" :) )
 
> But as I guess you're doing it differently now anyhow, I guess both are not
> affecting thhis solution now anyhow. :)

I think the solution for 6 will work just fine from a user PoV, this should be configurable (both to ease maintenance for us, and for non-mozilla installs.)
Comment 26 Robert Kaiser (not working on stability any more) 2012-04-13 16:48:41 PDT
(In reply to Robert Helmer [:rhelmer] from comment #25)
> This is good to know, thanks. We might need to change strategy in this case,
> let's discuss in irc.

Sure. I won't be around on IRC before Monday, probably, it's already Saturday and weekend around here. ;-)

> Otherwise builds for all kinds of other repositories show up (dev branches
> like "oak", ESR, etc - basically all the different types of dir in
> http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/ - repository is
> really not the 100% correct name in many cases, it's more like a "conceptual
> branch" :) )

Ah, it's "the part before the platfom in the FTP subdirectory name", which often corresponds to a repo. IIRC, releng calls it "branch".

> I think the solution for 6 will work just fine from a user PoV, this should
> be configurable (both to ease maintenance for us, and for non-mozilla
> installs.)

OK, good, that's what I care most about. :)
Comment 27 [github robot] 2012-04-16 06:31:53 PDT
Commit pushed to stage at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/e2c036a569e87ef462b1fe9c3f10ab873425279d
bug 740829 - temporary manual override for Fennec and FennecAndroid
Comment 28 Adrian Gaudebert [:adrian] 2012-04-16 07:00:15 PDT
Merged into master: https://github.com/mozilla/socorro/commit/6146b43bd139e301ee9888dcc30db7db674cd1cc

Also merged into our stage branch (see previous comment).
Comment 29 [:jberkus] Josh Berkus 2012-04-16 10:44:50 PDT
> 1) it's only updated one per day (currently it is once-per-hour)

From the database side, there's no particular obstacle to running update_products() more often.   Actually, I was under the impression that we were already running it 4 times a day.  No?

> So, since the product_builds service already hardcodes mozilla-specific
> config (repositories), I am temporarily adding hardcoding of Fennec*->mobile
> mappings as well.

good luck on the whole FennecXUL vs. FennecAndroid thing.  My suggestion is that you just label them all "Fennec" and don't worry about Android vs. XUL.  The if-then logic for that is annoying.

> For Socorro 7 (which will be released the week after 6), we can move all
> this stuff to the DB side and make it configurable so it won't break
> external installs.

You open a bug for that yet?
Comment 30 Matt Brandt [:mbrandt] 2012-04-18 09:14:32 PDT
Created attachment 616166 [details]
qa - verified

QA verified on stage - the nightly builds table appears to be correctly populated now for FennecAndroid - https://crash-stats.allizom.org/products/FennecAndroid/builds

All other products also contain nightly builds data except for Camino which is expected.
Comment 31 [github robot] 2012-04-21 09:01:34 PDT
Commits pushed to processor2012 at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/1d9eec10d50f35891b9d3d5382d093ae94a170cb
bug 740829 - temporary manual override for Fennec and FennecAndroid

https://github.com/mozilla/socorro/commit/6146b43bd139e301ee9888dcc30db7db674cd1cc
Merge pull request #500 from rhelmer/bug740829-android-nightly-report
Comment 32 [:jberkus] Josh Berkus 2012-04-23 16:59:19 PDT
Hmmmm.

Since we never opened a bug for the database work, I've completely forgotten what we were going to change to fix this.  I think I also needed a list of repositories, which I never got.

Help?
Comment 33 Robert Helmer [:rhelmer] 2012-04-23 18:13:41 PDT
(In reply to [:jberkus] Josh Berkus from comment #32)
> Hmmmm.
> 
> Since we never opened a bug for the database work, I've completely forgotten
> what we were going to change to fix this.  I think I also needed a list of
> repositories, which I never got.

Filed bug 748194

Just to clarify, this follow-up is not urgent - it's just a cleaner way of doing this (we shouldn't be building reports right on top of the raw tables, and it should be configurable and not require code changes to adjust).

Note You need to log in before you can comment on or make changes to this bug.