770019 - Some active versions are missing on Socorro (mostly Aurora, also Nightly)

Status: RESOLVED FIXED

(Socorro :: Database, task)

Description

[Robert Kaiser]

When looking through the "Products" tab of https://crash-stats.mozilla.com/admin/branch_data_sources, I realized yesterday that a hunch I had already before was true: We are missing some active versions on Socorro.

Feels like the FTP scraper missed out on something there.

Those are missing and should be there as we have build for them:
Fennec 15.0a2 (Aurora)
Fennec 16.0a1 (Nightly)
Thunderbird 15.0a2 (Aurora)
SeaMonkey 2.12a2 (Aurora)

Comment 1

[Robert Helmer [:rhelmer]]

Hm no errors in the FTP scraper log, but we did ship a change to the code it uses in Socorro 14. I'll take a look.

Comment 2

[Robert Kaiser]

Note that if it's an FTP scraper error, it must go a few weeks back, as those versions all appeared first on or around June 5 and had daily builds since then.

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to Robert Kaiser (:[e-mail]) from comment #2)
> Note that if it's an FTP scraper error, it must go a few weeks back, as
> those versions all appeared first on or around June 5 and had daily builds
> since then.

Hmm ok thanks, the change  I am thinking of went live June 27th so not likely not this. I'll figure out the issue anyway.

Comment 4

[Robert Helmer [:rhelmer]]

OK problem here is not scraper, but we filter based on the "repository" designator in the FTP directory name.

We are missing these entries in release_repositories:

comm_aurora
mozilla-aurora-android-xul

Comment 5

[Robert Kaiser]

What about mozilla-central-android-xul, as 16.0a1 is also missing for Fennec?

Comment 6

[Robert Helmer [:rhelmer]]

I think to fix we'll just need:

INSERT INTO release_repositories VALUES ('comm_aurora'), ('mozilla-aurora-android-xul'), ('mozilla-central-android-xul');
SELECT backfill_matviews('2012-06-05', '2012-06-30'); -- (is it necessary to backfill this far?)

mpressman and I are testing on staging now.

Comment 7

[Robert Helmer [:rhelmer]]

(In reply to Robert Kaiser (:[e-mail]) from comment #5)
> What about mozilla-central-android-xul, as 16.0a1 is also missing for Fennec?

Good catch, thanks (mentioned in comment 6)

Comment 8

[[:jberkus] Josh Berkus]

Rob,

That's all you need, correct.  Just insert into that table, it should work.

Comment 9

[Robert Helmer [:rhelmer]]

mpressman, can you run comment 6 on prod (after hours) when you have time (except to the current date instead of 06-300? Assuming it ran ok on stage (I remember it took a while to backfill)

Comment 10

[Matt Pressman [:mpressman]]

this has been kicked off

Comment 11

[Matt Pressman [:mpressman]]

this has been completed, it ran from 2012-06-05 - 2012-07-10

Comment 12

[Robert Kaiser]

(In reply to Robert Kaiser (:[e-mail]) from comment #0)
> Those are missing and should be there as we have build for them:
> Fennec 15.0a2 (Aurora)
> Fennec 16.0a1 (Nightly)

Those are there now.

> Thunderbird 15.0a2 (Aurora)

This is there but has the wrong channel:
Thunderbird 	15.0a2 	Release 	2012-06-04 	2012-08-06

> SeaMonkey 2.12a2 (Aurora)

This is still missing.

Some problem with "comm_aurora"?
Well, looking in http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/ and http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/ makes me realize it's really called "comm-aurora", with a dash, not an underscore...

I guess that's reason enough to reopen.

Comment 13

[[:jberkus] Josh Berkus]

So the repository name is fixed and TB 15.0a2 is now available as a product.  Do we need to backfill?

Comment 14

[Robert Kaiser]

(In reply to [:jberkus] Josh Berkus from comment #13)
> So the repository name is fixed and TB 15.0a2 is now available as a product.
> Do we need to backfill?

Let's go without a backfill, nobody from the SeaMonkey and Thunderbird communities has complained so far about those versions missing.

Comment 15

[Robert Kaiser]

This can be marked fixed (not sure if we should set a target milestone), as it's working correctly now. The new versions from this week's uplift have been picked up correctly as well.