Closed Bug 480081 Opened 15 years ago Closed 15 years ago

Generate partial & complete mars for nightly l10n builds on m-c

Categories

(Release Engineering :: General, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: nthomas, Assigned: armenzg)

References

Details

(Whiteboard: [l10n])

Attachments

(5 files, 13 obsolete files)

5.86 KB, patch
coop
: review+
Details | Diff | Splinter Review
7.03 KB, patch
coop
: review+
Details | Diff | Splinter Review
5.33 KB, patch
coop
: review+
Details | Diff | Splinter Review
999 bytes, patch
coop
: review+
Details | Diff | Splinter Review
656 bytes, patch
armenzg
: review+
Details | Diff | Splinter Review
We're just creating exe/zip/tar.bz2/dmg at the moment, but need complete mars to do another test in bug 459878. This is a mod to the system bsmedberg originally set up.
Priority: -- → P2
Need to
* configure with --enable-update-packaging, RepackFactory's extraConfigureArgs can help
* make in nsprpub, config and modules/libmar to get dist/host/mar
* set MOZ_MAKE_COMPLETE_MAR=1 when calling make installers-ab-CD in order to get a mar file
* locate complete mar's in objdir and adjust master.cfg's shipFiles to upload them
Attached patch "buildbotcustom" patch (obsolete) — Splinter Review
This is a patch against the fork of buildbotcustom
  http://hg.mozilla.org/users/bsmedberg_mozilla.com/buildbotcustom/
which adds a completeMar option to RepackFactory, then gets conditional-happy using it.
Attached patch master.cfg changes (obsolete) — Splinter Review
Changes to the buildbot config to support complete mar generation - basically just update the manifest of files to upload and flip the boolean.

These patches work well on Linux and Windows, where I've checked tar.bz2 vs mar file contents, and zip vs exe vs mar. Didn't let the files upload, but it should work. 

On mac, bm-l10n-pmac-01 doesn't have the build environment of our other slaves so configure gives you
  configure: error: Library requirements (libIDL-2.0 >= 0.8.0) not met; consider adjusting the PKG_CONFIG_PATH environment variable if your libraries are in a nonstandard prefix so pkg-config can find them.
If we want to go ahead with this (dependent on b3 start timing) then we'd need to fix this up by rsyncing over /opt/ from another slave (to get all macports stuff).
I suspect this is INVALID for bsmedberg's machines, and I don't need nightly complete mars for bug 459878 now that 3.1b3 got started. Pushing it back to the pool for triage against on our future plans for l10n.
Assignee: nthomas → nobody
No longer blocks: 459878
Status: ASSIGNED → NEW
This might be a good one for Armen to pick up. cc-ing Axel as well for his thoughts on l10n updates.
Priority: P2 → P3
Whiteboard: [l10n]
This surely blocks updates on the nightly channel, aka bug 449828.

We probably want different build paths for dep builds and nightlies for this, so that we don't waste time and bandwidth creating the mars all day, when we only want them for the nightly.

I'd personally block this on a property-dependent factory, haven't filed a bug on that yet.
Blocks: 449828
Armen and I discussed him taking this on yesterday, but he may not have time to get to it for a few weeks.
Assignee: nobody → armenzg
1) Summary updated: Wouldnt we also want partial mars or am I missing something? 

2) Before we can have people getting nightly updates like this, we need to know that they are getting updated to *working* builds - ie a build that is not missing any l10n entities. Should that be a blocking bug?
Summary: Generate complete mars for nightly 1.9.1 l10n builds → Generate partial & complete mars for nightly 1.9.1 l10n builds
Yes, and yes.
That'd be bug 478436 for compare-locales/l10n merge.
Depends on: 478436
Priority: P3 → P2
Priority: P2 → P3
Summary: Generate partial & complete mars for nightly 1.9.1 l10n builds → Generate partial & complete mars for nightly l10n builds
After the whole week in this bug I would like to give an update of where I am and what are the problems that I am encountering. I will do this first briefly and after into details. My apologies for throwing so much at once.

I have accomplished:
- automate with buildbot the generation of mar files (thanks nthomas!)
- create manually a complete.update.snippet (thanks bhearsum - contents of one at the bottom of this comment)

I need to:
- test if the manually created snippet if uploaded to aus2-stage (production server) will be able to generate *partial updates* (complete updates can be tested in the staging AUS server without problems). I cannot do this by myself due to my lack of knowledge nthomas and bhearsum could be helping me with this
- automate the generation of snippets

#######################

PROBLEMS:
The generation of snippets for en-US goes like this:
1) call the CreateCompleteUpdateSnippet step - http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#669
2) this class basically generates a text file "complete.update.snippet" by pulling information from propeties - http://mxr.mozilla.org/build/source/buildbotcustom/steps/updates.py#62
3) these properties are set through SetMozillaBuildProperties - http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#619
4) this class calls a LoggedRemoteCommand - http://mxr.mozilla.org/build/source/buildbotcustom/steps/misc.py#174 - and loads a bunch of properties after it is run - http://mxr.mozilla.org/build/source/buildbotcustom/steps/misc.py#183
5) the LoggedRemoteCommand (which leaves in buildbot instead of buildbotcustom) loads the following properties that will be later used to create the snippet:
  - buildid
  - appVersion
  - packageFilename
  - packageHash
  - packageSize
  - installerFilename
  - installerHash
  - installerSize
  - completeMarFilename
  - completeMarHash
  - completeMarSize
6) as I mentioned once loaded this properties the CreationCompleteUpdateSnippet can generate the snippet

PROPOSED SOLUTION
We already load properties from the build system through "make ident" which gives us the en-US's checkout changeset. For fennec we have extended this target to pull two properties at once (showing that multiple of them could be pulled - I have used this locally to pull the buildid) http://mxr.mozilla.org/mobile-browser/source/locales/Makefile.in#93

I propose that the build system should do more rather than relying on custom buildbot code (I am not proposing change to the en-US system). I would like to extend the make ident target (or have another target) to pull all the mentioned properties that I need to generate the snippets.

My second proposal would be to have the build system to generate the snippet for us but this has less rush than pulling the properties. Which our previous releasing product (tinderbox) knew how to do it as well - http://mxr.mozilla.org/mozilla/source/tools/tinderbox/post-mozilla-rel.pl#751

###########################

The needed properties according to generateSnippet (http://mxr.mozilla.org/build/source/buildbotcustom/steps/updates.py#58) are:
- downloadURL
- completeMarHash
- completeMarSize
- buildid
- appVersion

Contents of a snippet for the curious:
------------------
complete
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2009/05/2009-05-19-03-mozilla-central/firefox-3.6a1pre.en-US.mac.complete.mar
sha1
d6134eda47910ff0bb67aa21eb1da1709b345dd6
16953371
20090519032945
3.6a1pre
3.6a1pre
--------------------
Depends on: 495100
Regarding snippet generation for en-US, do we generate snippets for mozilla-central and mozilla-1.9.1? If yes, do we want updates for locales in both of these branches?
We surely provide snippets and updates for all differently versioned branches, and I think it would be good to have that for L10n nightlies as well.
Depends on: 496196
No longer depends on: 495100
A quick update, I have filed bug 496196 to have a python script that generates the snippets for a locale. If anyone is interested to extend it for using it for en-US nightly snippet generation feel welcome to bring the discussion there.

I will be now working on making nightly repackages to be uploaded to firefox/tinderbox-builds rather that firefox/nightly/latest-blah.

After that I should be posting the patches making modifications to buildbotcustom/process/factopry.py and buildbot-configs/mozilla2-staging/master-main.cfg to enable the l10n nightly updates (which needs the snippet generation and uploading to AUS)
I have run a test run in staging and the complete MAR file has been uploaded correctly to "latest":
http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-1.9.1-l10n/firefox-3.5pre.fr.linux-i686.complete.mar
Here is the snippet that was uploaded:
http://staging-stage.build.mozilla.org/aus2/build/0/Firefox/mozilla-1.9.1/Linux_x86-gcc3/20090605031224/fr/complete.txt
which has an incorrect URL:
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/2009/06/2009-06-05-03-mozilla-1.9.1/firefox-3.5pre.fr.linux-i686.complete.mar
since it should point to staging-stage and in this run I did not create any dated folder and after mozilla-1.9.1 there should be a "-l10n".

Here is the log of the repackage (BTW I am testing "fr" and only Linux since I have to do local patching in the slave for the python script):
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaStaging/1244227404.1244227470.28923.gz&fulltext=1

Tested:
* MAR uploaded to latest
* Snippet uploaded correctly

Missing:
* MAR uploaded to dated dir
* Rectify URL
Here is the dated folder:
http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/2009-06-05-03-mozilla-1.9.1-l10n/

I would give a log of it but the tbox page is gone. *sigh*

GLOBAL TODO:
- test repack-on-change scenario locally
- test repack-on-change scenario on staging-master to test proper uploading
- work on the python script to be called directly by buildbot rather than makefile
- fix the URL generated in the snippet
Just a few points of clarification:

* we're only targeting 3.5+ l10n nightlies for updates. We're explicitly *not* going to provide updates for 3.0 l10n nightlies.

* complete updates are easier to provide than partial updates, so it is likely that we will begin by providing complete updates, and then start providing partial updates sometime later. This will likely be in Q3, and may be dependent on improvements to the nightly update system as a whole (en-US included).
Depends on: 497653
Adding the dependency since I would like to see if the updates would work if downloaded from staging-stage before reaching production (Bug 497679)
Depends on: 497679
No longer depends on: 497653
Attachment #364240 - Attachment is obsolete: true
This is what has been running on staging-master properly but
I am not asking for review until I test that it works for the repackage-on-change scenario. The same goes with the previous patch.
Attachment #364237 - Attachment is obsolete: true
Attachment #383975 - Flags: review?(l10n)
Attachment #383975 - Flags: review?(ccooper)
Attachment #383340 - Attachment is obsolete: true
Attachment #383342 - Attachment is obsolete: true
Attachment #383976 - Flags: review?(l10n)
Attachment #383976 - Flags: review?(ccooper)
The previous two patches have been tested that they detect the repackages on change as well.
Attachment #383975 - Flags: review?(ccooper) → review+
Comment on attachment 383975 [details] [diff] [review]
moz2-staging - it enables l10n nightly updates

>-builders = Firefox\ mozilla-1.9.1\ linux\ l10n Firefox\ mozilla-1.9.1\ macosx\ l10n Firefox\ mozilla-1.9.1\ win32\ l10n
>+builders = Firefox\ mozilla-1.9.1\ linux\ l10n\ build Firefox\ mozilla-1.9.1\ macosx\ l10n\ build Firefox\ mozilla-1.9.1\ win32\ l10n\ build

Is there a reason behind the builder name change here (and elsewhere)?
Comment on attachment 383976 [details] [diff] [review]
buildbotcustom - it changes the factory to enable a nightly mode and adds the steps necessary to generate MAR and snippet files

>+            # this is a tad ugly because we need to python interpolation
>+            # as well as WithProperties

Change to "we need to do python" <- missing "do."

r+ with nit fixed.
Attachment #383976 - Flags: review?(ccooper) → review+
Comment on attachment 383976 [details] [diff] [review]
buildbotcustom - it changes the factory to enable a nightly mode and adds the steps necessary to generate MAR and snippet files

Just as a not, the factory.py change here will bitrot the patch from bug 494577 if it goes in earlier than that one.
(In reply to comment #24)
> (From update of attachment 383975 [details] [diff] [review])
> >-builders = Firefox\ mozilla-1.9.1\ linux\ l10n Firefox\ mozilla-1.9.1\ macosx\ l10n Firefox\ mozilla-1.9.1\ win32\ l10n
> >+builders = Firefox\ mozilla-1.9.1\ linux\ l10n\ build Firefox\ mozilla-1.9.1\ macosx\ l10n\ build Firefox\ mozilla-1.9.1\ win32\ l10n\ build
> 
> Is there a reason behind the builder name change here (and elsewhere)?
>
Yes, one of the builders will do the nightly l10n repackages and the one ending with "build" will do the repackages on change. The disadvantage is to have twice the mozilla-central and 1.9.1 checked out. This will be fixed when we have jhford's centralized souce checkout.
Comment on attachment 383975 [details] [diff] [review]
moz2-staging - it enables l10n nightly updates

>@@ -391,6 +399,33 @@
>                     'category': name,
>                 }
>                 c['builders'].append(mozilla2_l10n_nightly_builder)
>+                mozilla2_l10n_dep_factory = NightlyRepackFactory(
>+                    hgHost=HGHOST,
>+                    tree=branch['l10n_tree'],
>+                    project=branch['product_name'],
>+                    appName=branch['app_name'],
>+                    enUSBinaryURL=branch['enUS_binaryURL'],
>+                    nightly=False,
>+                    stageServer=STAGE_SERVER,
>+                    stageUsername=STAGE_USERNAME,
>+                    stageSshKey=STAGE_SSH_KEY,
>+                    repoPath=branch['repo_path'],
>+                    l10nRepoPath=branch['l10n_repo_path'],
>+                    buildToolsRepoPath=BUILD_TOOLS_REPO_PATH,
>+                    compareLocalesRepoPath=COMPARE_LOCALES_REPO_PATH,
>+                    compareLocalesTag=COMPARE_LOCALES_TAG,
>+                    buildSpace=2,
>+                    clobberURL=BASE_CLOBBER_URL,
>+                    clobberTime=clobberTime,
>+                )

Armen and I were discussing landing this morning, and what we really want here is a config toggle that we can use to enable/disable nightly updates entirely. That will allow us to land the buildbotcustom changes in advance of the make targets, and will provide a convenient switch for later (or for other projects that want to re-use this code). 

Something like, e.g.: BRANCHES['mozilla-central']['l10nNightlyUpdate'] = True
Attachment #383975 - Flags: review+ → review-
Comment on attachment 383976 [details] [diff] [review]
buildbotcustom - it changes the factory to enable a nightly mode and adds the steps necessary to generate MAR and snippet files

See comment #28 for details.
Attachment #383976 - Flags: review+ → review-
Comment on attachment 383975 [details] [diff] [review]
moz2-staging - it enables l10n nightly updates

>+                    createSnippet=branch['create_snippet'],

We're currently using the branch-wide flag create_snippet to determine whether we run the nightly update code for l10n. I think that's a good eventual goal, i.e. have en-US and l10n nightly updates on par with each other, but having an extra flag (l10nNightlyUpdates or some such) for now will allow us to do the staged landings we need in advance of the make targets.
If the parameter l10NightlyUpdates is False (which is by default) it uses the system that we have been using so far (no dated dirs being used)

If the parameter is True it does exactly what the previous patch did
Attachment #383976 - Attachment is obsolete: true
Attachment #384417 - Flags: review?(ccooper)
Attachment #383976 - Flags: review?(l10n)
the only difference with previous patch is to make it explicit the usage of l10nNightlyUpdate in the factory
Attachment #383975 - Attachment is obsolete: true
Attachment #384418 - Flags: review?(ccooper)
Attachment #383975 - Flags: review?(l10n)
Comment on attachment 384417 [details] [diff] [review]
buildbotcustom - it changes the factory to enable a nightly mode and adds the steps necessary to generate MAR and snippet files v3

>+        postUploadCmd =  ['post_upload.py']
>+
>+        if l10nNightlyUpdate:
>+            postUploadCmd += ['-i %(buildid)s',
>+                              '-p %s' % kwargs['project'],
>+                              '-b %s' % uploadDir]
>+
>+            self.env = {}
>+            if self.nightly:
>+                # We want to generate MAR files for the nightly scenario
>+                # and upload to 1) latest and 2) a dated directory
>+                self.env = {'MOZ_MAKE_COMPLETE_MAR': '1'}
>+                self.extraConfigureArgs = ['--enable-update-packaging']
>+                postUploadCmd += ['--release-to-latest',
>+                              '--release-to-dated']
>+            else:
>+                # For the repack-on-change scenario we just want to upload
>+                # to tinderbox dated directories
>+                postUploadCmd += ['--release-to-tinderbox-dated-builds']
>+            self.postUploadCmd = WithProperties(' '.join(postUploadCmd))
>+        else:
>+            self.postUploadCmd = 'post_upload.py ' + \
>+                                 '-p %s ' % kwargs['project'] + \
>+                                 '-b %s ' % uploadDir + \
>+                                 '--release-to-latest'

I'm not a fan of the factoring here. Can we move the "-p" and "-b" options outside the if/else since they are always used?

Also, let's use consistent syntax for setting self.postUploadCmd. There's no reason to start with "postUploadCmd =  ['post_upload.py']" outside the if clause if you are just going to concat a string in the else clause.
Attachment #384417 - Flags: review?(ccooper) → review-
Attachment #384418 - Flags: review?(ccooper) → review+
Attachment #384660 - Flags: review?(ccooper) → review+
Attachment #384418 - Flags: review?(catlee)
Attachment #384660 - Flags: review?(catlee)
Attachment #384418 - Flags: review?(catlee) → review+
Attachment #384660 - Flags: review?(catlee) → review+
Comment on attachment 385092 [details] [diff] [review]
moz2-production v0 - it enables l10n nightly updates (using l10nNightlyUpdates parameter)

Should set l10nNightlyUpdate for tracemonkey too to avoid key errors.
Attachment #385092 - Flags: review?(ccooper) → review-
Attachment #385092 - Attachment is obsolete: true
Attachment #385205 - Flags: review?(ccooper)
Attachment #385205 - Flags: review?(ccooper) → review+
Attachment #384418 - Flags: checked‑in? checked‑in?
Attachment #384660 - Flags: checked‑in? checked‑in?
Attachment #385205 - Flags: checked‑in? checked‑in?
Attachment #384660 - Flags: checked‑in? checked‑in? → checked‑in+ checked‑in+
Comment on attachment 384660 [details] [diff] [review]
buildbotcustom v4 - it changes the factory to enable a nightly mode and adds the steps necessary to generate MAR and snippet files

http://hg.mozilla.org/build/buildbotcustom/rev/1b4a558f099d
Attachment #384418 - Flags: checked‑in? checked‑in? → checked‑in+ checked‑in+
Attachment #385205 - Flags: checked‑in? checked‑in? → checked‑in+ checked‑in+
We actually managed to get the make target changes landed successfully on m-c today, so I've landed the config changes with l10nNightlyUpdates=True for m-c. We might as well take proper advantage of the downtime tomorrow by turning this on.

http://hg.mozilla.org/build/buildbot-configs/rev/c7d761556e15
sweet - I will have an eye over the weekend
We have to add a line on stage so we don't pile more than 2 days of dated l10n repackages
Attachment #384418 - Flags: checked‑in+ checked‑in+
Attachment #384660 - Flags: checked‑in+ checked‑in+
Attachment #385205 - Flags: checked‑in+ checked‑in+
This morning we hit minor problems when trying to deploy the patches and had to back them out.

The problem came from:
1) identToProperties for 1.9.1 won't work since the make ident patch has not landed in that tree. To avoid this until landing it I will leave fx_revision in place. The problem is that I tested the patches in slaves in which I had deployed manually the patches from the dependent bug. My bad.

2) one of the slaves had the mozilla-central folder but with nothing in it making the get en_US source step to fail. This was a coincidence. A bad clobbering from previous builds

3) l10n upload for the installers failed. This should have been caught in my testing of the post_upload.py default behaviour changes.

######

I have retested the scenarios and will soon repost the patches with the slight required changes.
Attachment #384418 - Attachment is obsolete: true
Attachment #385476 - Flags: review?(ccooper)
Comment on attachment 385477 [details] [diff] [review]
buildbotcustom v6 - it changes the factory to enable a nightly mode and adds the steps necessary to generate MAR and snippet files

- I have added a step for removing dist/update
- I have fixed the post_upload.py problem
- I have changed the name of addUpdateSteps() to uploadSnippet() since the MAR file and the snippet are generated when we do "make installers-foo"
Attachment #385476 - Flags: review?(ccooper)
Attachment #385477 - Flags: review?(ccooper)
added optional datedDirs parameter
Attachment #385477 - Attachment is obsolete: true
Attachment #385817 - Flags: review?(ccooper)
This has been running correctly in staging-master over the weekend.

mozilla-central with nightly updates and with dated dirs
mozilla-1.9.1 without nightly updates and without dated dirs (same as the old way)
Attachment #385205 - Attachment is obsolete: true
Attachment #385815 - Flags: review?(ccooper) → review+
Comment on attachment 385815 [details] [diff] [review]
moz2-staging v5 - it enables l10n nightly updates and dated dirs *only* for moz-central

Looks good, but I'm going to change 'datedDirs' to 'l10nDatedDirs' on landing for consistency.
Attachment #385817 - Flags: review?(ccooper) → review+
Attachment #386120 - Flags: review?(ccooper) → review+
Attachment #385815 - Flags: checked‑in+ checked‑in+
Attachment #385817 - Flags: checked‑in+ checked‑in+
Attachment #386120 - Flags: checked‑in+ checked‑in+
Yesterday we turned on the l10n nightly updates for mozilla-central. The MARs are being generated but the updates are not working. The complete.txt and patial.txt files are empty because the snippets uploaded have not been generated properly by generatesnippet.py. Product is missing from the URL pointing to the MAR files

The line that is missing the product is http://mxr.mozilla.org/mozilla-central/source/tools/update-packaging/generatesnippet.py#88

The URL points to:
http://ftp.mozilla.org/pub/mozilla.org/nightly/2009/07/2009-07-02-03-mozilla-central-l10n/firefox-3.6a1pre.fy-NL.mac.complete.mar
when it should point to:
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2009/07/2009-07-02-03-mozilla-central-l10n/firefox-3.6a1pre.fy-NL.mac.complete.mar

The empty snippets are in:
/opt/aus2/incoming/2/Firefox/mozilla-central/Darwin_Universal-gcc3/20090702031635
The snippets that have been uploaded by the slaves:
/opt/aus2/build/0/Firefox/mozilla-central/Darwin_Universal-gcc3/20090702031635

## beginning of snippet
complete
http://ftp.mozilla.org/pub/mozilla.org/nightly/2009/07/2009-07-02-03-mozilla-central-l10n/firefox-3.6a1pre.fy-NL.mac.complete.mar
sha1
1df337718442cb9e55b2f1f8a9abdad4d32472ae
17580636
20090702031635
3.6a1pre
3.6a1pre
## end of snippet
(In reply to comment #51)
> The line that is missing the product is
> http://mxr.mozilla.org/mozilla-central/source/tools/update-packaging/generatesnippet.py#88

Sounds to me like we should hand the script a --download-base-URL but you aren't doing that. We have DOWNLOAD_BASE_URL in the buildmaster's config.py and we should hand that over to the script.
Attachment #386732 - Flags: review?(ccooper) → review+
Comment on attachment 386732 [details] [diff] [review]
DOWNLOAD_BASE_URL was not being passed properly to generatesnippet.py

http://hg.mozilla.org/build/buildbotcustom/rev/2c8a4ab0c551
Attachment #386732 - Flags: checked‑in+ checked‑in+
Had to disable nightly update again:

http://hg.mozilla.org/build/buildbot-configs/rev/b8528f6d3703

The patcher script was getting confused with locales and we didn't really want to *start* debugging is at 7:00pm on Friday.

[6:23pm] nthomas: the problem is that it's trying to build an fr partial, and download an hr complete from 20090702
[6:24pm] nthomas: something extremely weird is going on
Depends on: 502612
Summary
#######
We are taking a step back and making sure the next time we try to enable on production we will be successful.

On bug 502612 we are going to set up the same system that is used on production to generate the partial updates (which is what was not working properly on last week's attempt). Everything run properly on the one run we did few weeks ago but this time it just did not work as expected.

Meanwhile Axel is getting landed the Makefile for 1.9.1 and I will be trying to get generatesnippet.py landed on 1.9.1.
I am adding the dependency on bug 489313 since once it lands it will allow us to call the generatesnippet.py on 1.9.1.

Tomorrow's downtime will enable l10n nightly updates on mozilla-central and once bug 489313 lands we will be able to enable it on 1.9.1
Depends on: 489313
Getting the patch ready for tomorrow's downtime.
Attachment #393699 - Flags: review?(armenzg)
Attachment #393699 - Flags: review?(armenzg) → review+
Attachment #393699 - Flags: checked-in+
Comment on attachment 393699 [details] [diff] [review]
Enable l10n nightly updates for mozilla-central in production

http://hg.mozilla.org/build/buildbot-configs/rev/28f1149c6eb8
The l10n nightly repacks with their complete MAR files are in:
* Mac - http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2009-08-10-03-mozilla-central-l10n/ 
* Win32 - http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2009-08-10-05-mozilla-central-l10n/
* Linux - http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2009-08-11-03-mozilla-central-l10n/

Mac and Win32 are made off the en-US nightly of the 10th since that was what there was when they were triggered after the downtime.

Tomorrow in these folders we should have the partial MAR files.
Do we have a cron job in place on stage to remove older dated l10n dirs ? There's plenty of free space there at the moment so there's time to find a nice solution.
The mentioned folders in comment 60 have their partial MAR files in them.

Partial updates are working and apply as expected.

Windows and Linux do update.
Mac does not update and the URL does not advertise any update.

For instance, if I have this URL https://aus2.mozilla.org/update/3/Firefox/3.6a2pre/20090810031114/Darwin_Universal-gcc3/af/nightly/Darwin%209.8.0/default/default/update.xml?force=1 I believe it should match to /opt/aus2/incoming/2/Firefox/mozilla-central/Darwin_Universal-gcc3/20090810031114/af in aus2-staging but it shows as if there was nothing even thought the snippets are there (complete.txt and partial.txt)
Depends on: 510080
I have filed as blocker the bug 510080 for the Mac l10n repackages not updating.

We also have 2 bugs related to turning on the l10 nightly updates but we won't be blocking on them. We will fix them without backing out the l10n nightly updates. Bugs 510042 and 509937.
No longer depends on: 510080
Follow up bugs and regresssions are usually added as dependent bugs, i.e., blocked by this one. Does it make sense to add those here?
(In reply to comment #64)
> Follow up bugs and regresssions are usually added as dependent bugs, i.e.,
> blocked by this one. Does it make sense to add those here?
>
I see what you mean. My thought in here was that if the Mac thing gets fixed and the updates would be *done* regardless if the other two came a day or two after.
Adding them since they are a regression (bug 510042) and a follow up (bug 509937).
Depends on: 510042, 509937
Depends on: 510080
Depends on: 510503
(In reply to comment #61)
> Do we have a cron job in place on stage to remove older dated l10n dirs ?
> There's plenty of free space there at the moment so there's time to find a nice
> solution.
>
I don't want to rely on our memory to do this. I have filed bug 510503
Status update:
- All platforms are working
- Bug 510503 tracks the cronjobs that have to be fine tuned
- Bug 510042 and bug 509937 will be fixed soon
We finished all the work on this bug to generate the partial and complete MAR files for mozilla-central.

The only work left is in the follow up bugs and enabling updates for 1.9.2.

Axel do the dependencies make a little more sense?
Can we resolve fixed and continue the work on the other bugs?
Blocks: 509937, 510042, 510503
No longer depends on: 509937, 510042, 510503
Yep, closing as FIXED.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Blocks: 511460
Summary: Generate partial & complete mars for nightly l10n builds → Generate partial & complete mars for nightly l10n builds on m-c
Blocks: 511901
Blocks: 511724
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.