468479 - mozilla2/ and mozilla2-staging/ Buildbot configs need to allow different mozconfigs for different repositories

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[[e-mail] (:bhearsum)]

This has become an issue now that we've turned on DWARF support in mozilla-central but not mozilla-1.9.1. We're going to need support for this at some point anyways, so this is a good opportunity to fix it.

Comment 1

[[e-mail] (:bhearsum)]

Attachment: different mozconfigs for different repositories - buildbot side

This is a pretty confusing patch, I'm sure. Mostly, it just reorganizes the mozconfigs and makes use of symlinks where possible to reduce the maintenance load.

For config.py/master.cfg we define a 'mozconfig' per platform - which is the path to the directory containing the mozconfig. mobile_master.py is updated, as well.

The rest of this patch is simply reorganizing the structure of the mozconfigs. It now looks as follows:
$platform/$repo/$type

There's symlinks for mozilla-1.9.1 and tracemonkey on linux and windows which point to mozilla-central for the respective platforms. On Mac, we can only symlink the 'debug' configs because we're currently still using -gstabs+ on tm and mozilla-1.9.1.

I gave this a good go in staging and AFAICT every different combination of $platform, $repo and $type pulls from the right place.

Associated factory.py patch incoming.

Comment 2

[[e-mail] (:bhearsum)]

Attachment: factory.py change to support new mozconfig paths

This is pretty simple. We're just using the passed in mozconfig path in MercurialBuildFactory to find the mozconfig, rather than constructing it. In the Unittest factory we simply have to adjusting the platform/mozconfig map.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 352165 [details] [diff] [review]
factory.py change to support new mozconfig paths

r+ with a fix required.

>Index: process/factory.py
> class MercurialBuildFactory(BuildFactory):
...
>@@ -167,8 +167,8 @@
>         # we need that to figure out which mozconfig to use
>         # but for other purposes we only need to know linux/win32/macosx
>         # platform can be things like: linux, win32-debug, macosx-release, etc.
>-        self.mozconfig = 'configs/%s/%s/mozconfig' % (self.configSubDir,
>-                                                      platform)
>+        self.mozconfig = 'configs/%s/%s' % (self.configSubDir, mozconfig)
>+

I think you're going to get self.mozconfig set to
   configs/mozilla2-staging/linux/mozilla-central/nightly
instead of 
   configs/mozilla2-staging/linux/mozilla-central/nightly/mozconfig
and it isn't appended later. Please put it back on checkin.

The comment also needs fixing now that platform is not used.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 352163 [details] [diff] [review]
different mozconfigs for different repositories - buildbot side

r+ with some minor fixes needed.

>diff -r b76bbdd5af57 mozilla2-staging/config.py
>+BRANCHES['mozilla-central']['platforms']['linux64']['mozconfig'] = 'linux/mozilla-central/nightly'
>+BRANCHES['mozilla-1.9.1']['platforms']['linux64']['mozconfig'] = 'linux/mozilla-1.9.1/nightly'

Should use the linux64 dir here, it's symlinked back to linux anyway.

>diff -r b76bbdd5af57 mozilla2-staging/linux/mozilla-central/xulrunner/mozconfig
>--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
>+++ b/mozilla2-staging/linux/mozilla-central/xulrunner/mozconfig	Tue Dec 09 13:17:15 2008 -0500

I was thinking the new scheme will break when we come to do XULRunner release builds, but perhaps their mozconfig's won't actually differ. That's break in a style or consistency sense, since you can put whatever you like in config.py for the mozconfig definition. Doubt another dir level for app is worth it for us.

>diff -r b76bbdd5af57 mozilla2-staging/win32-xulrunner/mozconfig
>--- a/mozilla2-staging/win32-xulrunner/mozconfig	Mon Dec 08 09:39:30 2008 -0500
>+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000

This one is never added again.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Please remove the ppc prebinding line from the mac xulrunner config too.

Comment 6

[[e-mail] (:bhearsum)]

Attachment: factory.py patch, as landed

cvs commit: Examining .
Checking in factory.py;
/cvsroot/mozilla/tools/buildbotcustom/process/factory.py,v  <--  factory.py
new revision: 1.54; previous revision: 1.53
done

Comment 7

[[e-mail] (:bhearsum)]

Attachment: buildbot config patch, updated

I landed this and backed it out because it blew up :(.

Comment 8

[[e-mail] (:bhearsum)]

Comment on attachment 352532 [details] [diff] [review]
factory.py patch, as landed

This got backed out due to bustage.

Comment 9

[[e-mail] (:bhearsum)]

It looks like mq busted my patch right before I landed it. It doesn't properly handle symlinks. I'll have to try this again another day.

Comment 10

[[e-mail] (:bhearsum)]

Comment on attachment 352532 [details] [diff] [review]
factory.py patch, as landed

This is in for real this time. The buildbot config patch will land tomorrow, so we don't screw up the location of mozconfigs in the meantime.

Checking in factory.py;
/cvsroot/mozilla/tools/buildbotcustom/process/factory.py,v  <--  factory.py
new revision: 1.58; previous revision: 1.57
done

Comment 11

[[e-mail] (:bhearsum)]

Comment on attachment 352534 [details] [diff] [review]
buildbot config patch, updated

changeset:   591:6def3b56848b
and
changeset:   593:fe260d5bd948

Comment 12

[[e-mail] (:bhearsum)]

This change took fine today.

Comment 13

[[e-mail] (:bhearsum)]

The symlinks ended up busting win32. I had to switch them out for real copies for the win32/ mozconfigs.