Closed Bug 748194 Opened 8 years ago Closed 8 years ago

product_versions should only include "official" repos from releases_raw

Categories

(Socorro :: Database, task)

task
Not set

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: rhelmer, Assigned: jberkus)

Details

Attachments

(1 file)

The Nightly Builds report (such as https://crash-stats.mozilla.com/products/Firefox/builds) is built right now by querying releases_raw, with the following rules hardcoded:

1) ignore "unofficial" builds, in practice this means:
   WHERE repository IN ('mozilla-central', 'mozilla-1.9.2',
                        'comm-central', 'comm-1.9.2',
                        'comm-central-trunk', 'mozilla-central-android')

2) mapping Fennec* -> "mobile"

We should build this on top of product_versions instead. It would need to be able to do something like #1 above (already does #2), and it should be done in a way that's configurable for non-Mozilla users (perhaps a mapping table?)
Yes, I'll be creating a mapping table called "release_repositories"
So, I would prefer to handle this by having a complete list of valid release repositories.  However, the list above applies only to nightly build repos, and not to other channels.  So, how difficult is it going to be to get a complete list of real release repositories?
Complete list of repositories in the database since March 1, excluding the -debug repos, is here: http://mozilla.privatepaste.com/69d9a003cd (link will expire in 1 month).

Looks like we need to add "mozilla-release", "mozilla-beta", "mozilla-aurora", "mozilla-aurora-android", "mozilla-esr10", "mozilla-esr10-android" at least, and possibly a few others.

Who would know this stuff?
I'm still a bit weary of calling this "repositories" as it's not a 1:1 match to actual code repositories. RelEng calls the same thing a "branch" identifier, maybe that makes more sense.
Robert,

The issue is that we called something else "branch" back in the oldTCBS days, and that's not entirely purged from the code & UI yet.  So I don't think renaming "repository" to "branch" would help the confusion.

More importantly, we'd really need to rename the field, which means changing the code which depends on it.
Done and pull request created.  Waiting on review to check into crash-stats-dev.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → 8
Reopening this because QA pointed out it lacks adequate tests.  Bumping to 9.0.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Target Milestone: 8 → 9
Still not getting correct results from this.  Since the consequences of excluding a legitimate build is data loss, wheras the current situation just has some unecessary noise, I'm bumping this to 10.0.  

Updated approach will include storing the list of repositories for each build in the product_version_builds table, so that we can audit it properly.
Target Milestone: 9 → 10
bumping back to 9.0, since we've just pushed 9.0 back a week.
Target Milestone: 10 → 9
Assignee: nobody → josh
nhirata,

Attached is a before-and-after set of CSV files for Fennec and FA releases between April 15 and April 30.  I'd like you to take a look at these and make sure I'm filtering out exactly the right set of releases.
Should this be RESOLVED FIXED?  Or moved out?  I'm assuming the latter.
Target Milestone: 9 → 10
Gah, I think this change went into 9.0 without getting a response back from nhirata.

Nhirata, please check those lists ASAP and let us know if they're correct!
(In reply to [:jberkus] Josh Berkus from comment #12)
> Gah, I think this change went into 9.0 without getting a response back from
> nhirata.

Yes if it was in the upgrade.sh then it went out. We should double-check that bugs mentioned in upgrade.sh are closed in the future, and also try not to have things in there that aren't ready ;)
Target Milestone: 10 → 9
Robert,

Yeah, I'm wondering if we need separate stage and prod branches.  The problem is that I can't actually test DB code unless it's in the stage branch, which means at that point I need to take action to revoke it.  Which means that in cases like this, if it passes build and execution tests, it goes in even if the result it's producing is untested.

Can you see if you can actually get nhirata to review the results?  They may actually be correct.
Working w/ jberkus;  I think we're set.

Note for how to check:
- Mobile is only releasing 13 on XUL for beta from now on.  It will not change in release numbers.
- ESR build will be 10.x for XUL as well.

Both are going to have candidate builds:
ex. for ESR build : http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/10.0.4esr-candidates/build2/linux-android_info.txt
 
ex. for XUL beta build : http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/13.0b3-candidates/build1/android-xul_info.txt

Nightly and Aurora Official builds can be checked by looking at :
http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/
and comparing the build id versus the directory names

ie : 2012-04-30-04-20-07-mozilla-aurora-android/ will match build id 20120430042007 for aurora (14.0a2)
Status: REOPENED → RESOLVED
Closed: 8 years ago8 years ago
Resolution: --- → FIXED
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.