451225 - master.cfg changes to run in *parallel* L10n repackages in 1.9 while doing a *release*

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Armen [:armenzg]]

When we do releases and we reach the "buildBuildFactory"
We build the en-US and we then do all repackages all in the same machine.

If we want take advantage of the parallization, we have two create a buildL10nFactory that will take care of splitting the work between slaves and therefore we will able to shorten the time that a release take

How much can we shorten it? It depends on how many slaves get involved specially in the windows builder

Comment 1

[Armen [:armenzg]]

I will have to write a DependentL10n scheduler (which I have done before) and remove the steps:
- 'perl', './release', '-e', '-o', 'Repack'

What do I do with these steps??:
- 'perl', './release', '-a', '-o', 'Repack' 
- 'perl', './release', '-p', '-o', 'Repack'
What do they do? My code for nightly builds pushes the packages to the ftp server. I went over the code but I am not sure what does it get done in there
------

Thinking out aloud about the change:
- buildBuildFactory starts
- linux and mac finish their en-US builds and wait until windows finishes
- windows finishes
- the DependentL10n scheduler triggers the buildBuildL10nFactory which starts handing locales to available slaves
- up to here we have made the linux and the mac machine wait before starting the L10n repackages - in the current way of doing things they wait after doing their own l10n repackages and wait all the way until the windows finishes all of its work before starting the signing process

FINAL NOTE:
- The time we would save in a release would be the difference between the time that it takes a windows single machine to do the l10n repackages and the time time that it will take to X number of windows slaves
- I mention the windows machine since it is the one that has the worst performance

Comment 2

[Armen [:armenzg]]

Attachment: it removes the Repack steps of bootstrap by a Dependent scheduler that trigers L10n repackages in parallel

I have not been able to run a fake release in staging 1.9 but this are the changes that should allows us to run L10n repackages in parallel
I will have another try tomorrow to do the release but if get guided would be better

This is blocked until the nightly builds scenario gets reviewed and approved (bug 451088)

Comment 3

[[e-mail] (:bhearsum)]

(In reply to comment #1)
> I will have to write a DependentL10n scheduler (which I have done before) and
> remove the steps:
> - 'perl', './release', '-e', '-o', 'Repack'
> 
> What do I do with these steps??:
> - 'perl', './release', '-a', '-o', 'Repack' 
> - 'perl', './release', '-p', '-o', 'Repack'

If your code pushes to the right place on stage to begin with, eg, /home/ftp/pub/firefox/nightly/2008-07-24-20-firefox3.0.2-l10n/, these steps will work just fine.

All Repack::Push (-p) does is copy the files from there to the candidates directory. Repack::Announce (-a) sends us an e-mail.

Comment 4

[Armen [:armenzg]]

(In reply to comment #3)
> (In reply to comment #1)
> > I will have to write a DependentL10n scheduler (which I have done before) and
> > remove the steps:
> > - 'perl', './release', '-e', '-o', 'Repack'
> > 
> > What do I do with these steps??:
> > - 'perl', './release', '-a', '-o', 'Repack' 
> > - 'perl', './release', '-p', '-o', 'Repack'
> 
> If your code pushes to the right place on stage to begin with, eg,
> /home/ftp/pub/firefox/nightly/2008-07-24-20-firefox3.0.2-l10n/, these steps
> will work just fine.
> 
I thought it would pull from latest-mozilla1.9.0-l10n rather than a dated folder
currently my code pushes to latest-mozilla.19-l10n
Where is it calculated from which dated folder to pull from?

Comment 5

[Robert Helmer [:rhelmer]]

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #1)
> > > I will have to write a DependentL10n scheduler (which I have done before) and
> > > remove the steps:
> > > - 'perl', './release', '-e', '-o', 'Repack'
> > > 
> > > What do I do with these steps??:
> > > - 'perl', './release', '-a', '-o', 'Repack' 
> > > - 'perl', './release', '-p', '-o', 'Repack'
> > 
> > If your code pushes to the right place on stage to begin with, eg,
> > /home/ftp/pub/firefox/nightly/2008-07-24-20-firefox3.0.2-l10n/, these steps
> > will work just fine.
> > 
> I thought it would pull from latest-mozilla1.9.0-l10n rather than a dated
> folder
> currently my code pushes to latest-mozilla.19-l10n
> Where is it calculated from which dated folder to pull from?

The folder that the build was uploaded to is determined by parsing the log file from the build process (e.g. Tinderbox):

http://mxr.mozilla.org/mozilla/source/tools/release/Bootstrap/Step/Repack.pm#202
http://mxr.mozilla.org/mozilla/source/tools/release/MozBuild/TinderLogParse.pm#75

Comment 6

[Armen [:armenzg]]

(In reply to comment #5)
> (In reply to comment #4)
> > (In reply to comment #3)
> > > (In reply to comment #1)
> > > > I will have to write a DependentL10n scheduler (which I have done before) and
> > > > remove the steps:
> > > > - 'perl', './release', '-e', '-o', 'Repack'
> > > > 
> > > > What do I do with these steps??:
> > > > - 'perl', './release', '-a', '-o', 'Repack' 
> > > > - 'perl', './release', '-p', '-o', 'Repack'
> > > 
> > > If your code pushes to the right place on stage to begin with, eg,
> > > /home/ftp/pub/firefox/nightly/2008-07-24-20-firefox3.0.2-l10n/, these steps
> > > will work just fine.
> > > 
> > I thought it would pull from latest-mozilla1.9.0-l10n rather than a dated
> > folder
> > currently my code pushes to latest-mozilla.19-l10n
> > Where is it calculated from which dated folder to pull from?
> 
> The folder that the build was uploaded to is determined by parsing the log file
> from the build process (e.g. Tinderbox):
> 
> http://mxr.mozilla.org/mozilla/source/tools/release/Bootstrap/Step/Repack.pm#202
> http://mxr.mozilla.org/mozilla/source/tools/release/MozBuild/TinderLogParse.pm#75
> 
Thanks!

I can see that the folder name is generated in here:
http://mxr.mozilla.org/mozilla/source/tools/tinderbox/post-mozilla-rel.pl#1397

This sucks! :S I will extract the logic...

Comment 7

[Axel Hecht [:Pike]]

Is grabbing from a time-stamped directory and pushing there a priority task or a "nice to have" addition?

Comment 8

[Axel Hecht [:Pike]]

Comment on attachment 334605 [details] [diff] [review]
it removes the Repack steps of bootstrap by a Dependent scheduler that trigers L10n repackages in parallel

Regarding the steps here, I don't think we need to check out either mozilla/browser/locales nor nsprpub initially, but that depends on the mozconfig. Which one are you using?

Why do the check-out of l10n in a separate buildstep instead of l10n-checkout? Might be a copy of a bug that I did back in the days.

No idea why installers-foo needs PKG_DMG_SOURCE set.

Comment 9

[[e-mail] (:bhearsum)]

(In reply to comment #7)
> Is grabbing from a time-stamped directory and pushing there a priority task or
> a "nice to have" addition?
> 

It's a requirement for releases.

Comment 10

[Armen [:armenzg]]

I haven't had time to work on this, putting back into the pool

Comment 11

[Axel Hecht [:Pike]]

I consider this to be FIXED.

Comment 12

[Chris Cooper [:coop] (he/him)]

Moving closed Future bugs into Release Engineering in preparation for removing the Future component.