838690 - No Daily builds on 02-06-2013 found

Status: VERIFIED FIXED

(Release Engineering :: General, defect)

Description

[Tony Chung [:tchung]]

Im guessing tinderbox bustage.   but there's no daily builds or updates found for today: 02-06-2013.

https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-b2g18-unagi/2013/02/

Comment 1

[Staś Małolepszy :stas]

Hal mentioned on IRC that this might have been caused by me landing changes to locales/languages_dev.json which is used in those builds to define which locales are available.  I added 11 shira locales but I didn't know that I should have first asked Hal to mirror these locales' Hg repos to git.

https://github.com/mozilla-b2g/gaia/commit/fd41681a3b52f2be790b257cbee9fe9c2fc2732c#diff-4

CC'ing Hal so that he can correct me if I got something wrong.

Comment 2

[Aki Sasaki (not active)]

That would do it.

Comment 3

[Aki Sasaki (not active)]

Otoro is changing upload locations: bug 821754.  It's green, but is pointing to languages_basecamp.json so would be unaffected by comment 1.

Both unagi nightlies from b2g18 died on "FATAL - Giving up on l10n git revision for 9fabe77b794d.", so this is indeed the culprit.

https://tbpl.mozilla.org/php/getParsedLog.php?id=19493866&tree=Mozilla-B2g18&full=1#error0
https://tbpl.mozilla.org/php/getParsedLog.php?id=19493883&tree=Mozilla-B2g18&full=1

Comment 4

[Aki Sasaki (not active)]

We can:

* point unagi at languages_basecamp.json on b2g18
** quick patch
** needs reconfig
** may affect QA testing
* remove those locales from languages_dev.json until they're ready
** this is probably the fastest fix
** not sure what this blocks
* update the sync + mapper portion, and leave unagi b2g18 broken until this is done.
** slowest fix
** we may not have nightlies for a day or two

Comment 5

[Aki Sasaki (not active)]

I think this became an issue yesterday because we finally started adding l10n git revisions to the source manifest for b2g18 unagi (we had been doing it already for otoro):
http://hg.mozilla.org/releases/mozilla-b2g18/rev/dd2c9fb8bcf0

So a fourth option would be to stop doing that until sync+mapper is updated.

Comment 6

[Staś Małolepszy :stas]

Is bug 838551 going to help?

Comment 7

[Aki Sasaki (not active)]

Sure, that's option 3 above.
Hal says a possible ETA is 2 hours; if that's acceptable we can go with that option and respin nightlies afterwards.

Comment 8

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Now that we know the root cause, moving to correct component.

Comment 9

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

interesting... so the eng build isn't affected by l10n at all?  Good to keep in mind.

Comment 10

[Aki Sasaki (not active)]

Right.
I don't think the eng build is distributed to partners; or if it is, we aren't translating the hg shas to git shas, so it bypasses this problem:

http://ftp.mozilla.org/pub/mozilla.org/b2g/manifests/1.0.1/2013-02-06-07/source_unagi-eng_2013-02-06-07.xml

Comment 11

[Alex Keybl [:akeybl]]

Let's go with the fastest fix (option 2, returning to yesterday's state) and re-spin since 2 hours is only an ETA for now.

Then let's work on bug 838551 in parallel.

Comment 12

[Aki Sasaki (not active)]

Backed out of v1-train only: https://github.com/mozilla-b2g/gaia/commit/9f5a4a9f6d321dbb4805f1f98623e770b0e89698

Once that replicates to hg.m.o, I will respin b2g18 nightlies.

Comment 13

[Aki Sasaki (not active)]

Replicated:
https://hg.mozilla.org/integration/gaia-v1-train/rev/076c8c1cf1f4

I kicked off new mozilla-b2g18 nightlies.

Comment 14

[Tony Chung [:tchung]]

(In reply to Aki Sasaki [:aki] from comment #13)
> Replicated:
> https://hg.mozilla.org/integration/gaia-v1-train/rev/076c8c1cf1f4
> 
> I kicked off new mozilla-b2g18 nightlies.

Thanks for the quick fix Aki!

Comment 15

[Aki Sasaki (not active)]

The builds went green, and I see them on pvtbuilds.

We can either resolve this bug and track bug 823478 for remaining work, or leave this bug open and lower severity.  I'm personally fine with either approach.

Comment 16

[Aki Sasaki (not active)]

(In reply to Aki Sasaki [:aki] from comment #15)
> We can either resolve this bug and track bug 823478 for remaining work, or
> leave this bug open and lower severity.  I'm personally fine with either
> approach.

Er, that should read bug 838551.

Comment 17

[Aki Sasaki (not active)]

Let's do that. Resolving since we have a short term workaround.
Bug 838551 for the medium-term fix.
A process for adding new locales for the long-term fix.

Comment 18

[Tony Chung [:tchung]]

verified working again.