578749 - turn on nightly l10n repacks for 64-bit linux, mac on mozilla-central & mozilla-2.0

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Comment 2

[Rail Aliiev [:rail]]

Attachment: Enable l10n repacks for linux64 and macosx64 (mozilla-central and mozilla-2.0)

Comment 3

[Armen [:armenzg]]

Comment on attachment 457786 [details] [diff] [review]
Enable l10n repacks for linux64 and macosx64 (mozilla-central and mozilla-2.0)

Is really that it? wow

BTW when you did run this through staging did both linux64 and osx10.6 succeed?
Which binaries are uploaded? I can't see any in here.
http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central-l10n/?C=N;O=A

Comment 4

[Rail Aliiev [:rail]]

(In reply to comment #3)
> Comment on attachment 457786 [details] [diff] [review]
> Enable l10n repacks for linux64 and macosx64 (mozilla-central and mozilla-2.0)
> 
> Is really that it? wow

Heh. At least AFAIK. :)

> BTW when you did run this through staging did both linux64 and osx10.6 succeed?
> Which binaries are uploaded? I can't see any in here.
> http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central-l10n/?C=N;O=A

I tried it for the 3.7a (iirc a4, a5 and a7 :) ) release builds which were cleaned up later for a real release (without l10n). I'd like to test this patch during the weekend in staging if it looks good to you.

Comment 5

[Armen [:armenzg]]

Looks good to me!

Comment 6

[Rail Aliiev [:rail]]

(In reply to comment #3)
> Comment on attachment 457786 [details] [diff] [review]
> Enable l10n repacks for linux64 and macosx64 (mozilla-central and mozilla-2.0)
> 
> Is really that it? wow

Not really. :(

NightlyBuildFactory.addCreatePartialUpdateSteps() fails in get_previous_mar_filename if there is no _previous_ complete mar on stage.

There are some ways to go.

1) The easiest work around. Set create_partial_l10n to False for mozilla-central just for one nightly build. The downside of this that we'll lose partial mars for existing l10n nightly builds (linux, macosx and win32) for one nightly.

2) Introduce create_partial_l10n_platforms variable (set of platforms), adapt NightlyBuildFactory.addCreatePartialUpdateSteps() so that it will produce partials only for preset platforms. We need to set this variable to ("linux", "macosx", "win32") firsts, then (after at least one nightly build) to ("linux", "macosx", "win32", "linux64", "macosx64").

3) NightlyBuildFactory.addCreatePartialUpdateSteps() shouldn't fail while getting  previous mar and use a dummy mar (empty mar, zero size mar, need to investigate) as a previous one.

Any other ideas how to enable l10n repacks smoothly?

Comment 7

[Axel Hecht [:Pike]]

Not smoothly is the answer. There's a bug, but I can't find it.

I guess the standard way right now is to copy over en-US mars, and have "funky" incrementals for the first run.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Bug 573999 was resolved recently, missing complete mars should just work now.

Comment 9

[Chris Cooper [:coop] (he/him)]

(In reply to comment #8)
> Bug 573999 was resolved recently, missing complete mars should just work now.

Yes, you should get a warning on your first run, but subsequent runs will be green.

Note: this code is still tagged as buildbot-0.8.0 in hg since not all our masters/slaves are running 0.8.0 yet.

Comment 10

[Rail Aliiev [:rail]]

(In reply to comment #9)
> Note: this code is still tagged as buildbot-0.8.0 in hg since not all our
> masters/slaves are running 0.8.0 yet.

Ah, oh. My bad, I started testing on moz-master2, as usual, instead of builder-master2. :)

Switching to buildbotcustom/buildbot-0.8.0 helped, but:

  File "/builds/buildbot/builder_master2/master.cfg", line 36, in <module>
    branchObjects = generateBranchObjects(BRANCHES[branch], branch)
  File "/tools/buildbotcustom2/buildbotcustom/misc.py", line 886, in generateBranchObjects
    'slavenames': config['l10n_slaves'][platform],
KeyError: 'macosx64'

Instead of "macox64" we use "macosx-snow". To minimize changes I added SLAVES['macosx64' = MAC_SNOW_MINIS to localconfig.py and the corresponding entry for SlavePasswords in BuildSlaves.py. With these 2 changes the first repack ran fine, warning issued, but the build in whole didn't stop and produced needed binaries/snippets. (Second run results will be available soon).

Is there any reason to use "macosx-snow" instead of "macox64"? According to the used (silent) name convention we don't duplicate platform names, and it should be "macox64" IMO.

Comment 11

[Rail Aliiev [:rail]]

Comment on attachment 457786 [details] [diff] [review]
Enable l10n repacks for linux64 and macosx64 (mozilla-central and mozilla-2.0)

http://hg.mozilla.org/build/buildbot-configs/rev/486cf48f32f8