mozilla2/ and mozilla2-staging/ Buildbot configs need to allow different mozconfigs for different repositories

RESOLVED FIXED

Status

Release Engineering
General
P2
normal
RESOLVED FIXED
9 years ago
5 years ago

People

(Reporter: bhearsum, Assigned: bhearsum)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 2 obsolete attachments)

(Assignee)

Description

9 years ago
This has become an issue now that we've turned on DWARF support in mozilla-central but not mozilla-1.9.1. We're going to need support for this at some point anyways, so this is a good opportunity to fix it.
(Assignee)

Updated

9 years ago
Priority: -- → P2
(Assignee)

Comment 1

9 years ago
Created attachment 352163 [details] [diff] [review]
different mozconfigs for different repositories - buildbot side

This is a pretty confusing patch, I'm sure. Mostly, it just reorganizes the mozconfigs and makes use of symlinks where possible to reduce the maintenance load.

For config.py/master.cfg we define a 'mozconfig' per platform - which is the path to the directory containing the mozconfig. mobile_master.py is updated, as well.

The rest of this patch is simply reorganizing the structure of the mozconfigs. It now looks as follows:
$platform/$repo/$type

There's symlinks for mozilla-1.9.1 and tracemonkey on linux and windows which point to mozilla-central for the respective platforms. On Mac, we can only symlink the 'debug' configs because we're currently still using -gstabs+ on tm and mozilla-1.9.1.

I gave this a good go in staging and AFAICT every different combination of $platform, $repo and $type pulls from the right place.

Associated factory.py patch incoming.
Attachment #352163 - Flags: review?(nthomas)
(Assignee)

Comment 2

9 years ago
Created attachment 352165 [details] [diff] [review]
factory.py change to support new mozconfig paths

This is pretty simple. We're just using the passed in mozconfig path in MercurialBuildFactory to find the mozconfig, rather than constructing it. In the Unittest factory we simply have to adjusting the platform/mozconfig map.
Attachment #352165 - Flags: review?(nthomas)
Comment on attachment 352165 [details] [diff] [review]
factory.py change to support new mozconfig paths

r+ with a fix required.

>Index: process/factory.py
> class MercurialBuildFactory(BuildFactory):
...
>@@ -167,8 +167,8 @@
>         # we need that to figure out which mozconfig to use
>         # but for other purposes we only need to know linux/win32/macosx
>         # platform can be things like: linux, win32-debug, macosx-release, etc.
>-        self.mozconfig = 'configs/%s/%s/mozconfig' % (self.configSubDir,
>-                                                      platform)
>+        self.mozconfig = 'configs/%s/%s' % (self.configSubDir, mozconfig)
>+

I think you're going to get self.mozconfig set to
   configs/mozilla2-staging/linux/mozilla-central/nightly
instead of 
   configs/mozilla2-staging/linux/mozilla-central/nightly/mozconfig
and it isn't appended later. Please put it back on checkin.

The comment also needs fixing now that platform is not used.
Attachment #352165 - Flags: review?(nthomas) → review+
Attachment #352163 - Flags: review?(nthomas) → review+
Comment on attachment 352163 [details] [diff] [review]
different mozconfigs for different repositories - buildbot side

r+ with some minor fixes needed.

>diff -r b76bbdd5af57 mozilla2-staging/config.py
>+BRANCHES['mozilla-central']['platforms']['linux64']['mozconfig'] = 'linux/mozilla-central/nightly'
>+BRANCHES['mozilla-1.9.1']['platforms']['linux64']['mozconfig'] = 'linux/mozilla-1.9.1/nightly'

Should use the linux64 dir here, it's symlinked back to linux anyway.

>diff -r b76bbdd5af57 mozilla2-staging/linux/mozilla-central/xulrunner/mozconfig
>--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
>+++ b/mozilla2-staging/linux/mozilla-central/xulrunner/mozconfig	Tue Dec 09 13:17:15 2008 -0500

I was thinking the new scheme will break when we come to do XULRunner release builds, but perhaps their mozconfig's won't actually differ. That's break in a style or consistency sense, since you can put whatever you like in config.py for the mozconfig definition. Doubt another dir level for app is worth it for us.

>diff -r b76bbdd5af57 mozilla2-staging/win32-xulrunner/mozconfig
>--- a/mozilla2-staging/win32-xulrunner/mozconfig	Mon Dec 08 09:39:30 2008 -0500
>+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000

This one is never added again.
Please remove the ppc prebinding line from the mac xulrunner config too.
(Assignee)

Comment 6

9 years ago
Created attachment 352532 [details] [diff] [review]
factory.py patch, as landed

cvs commit: Examining .
Checking in factory.py;
/cvsroot/mozilla/tools/buildbotcustom/process/factory.py,v  <--  factory.py
new revision: 1.54; previous revision: 1.53
done
Attachment #352165 - Attachment is obsolete: true
Attachment #352532 - Flags: checked‑in+
(Assignee)

Comment 7

9 years ago
Created attachment 352534 [details] [diff] [review]
buildbot config patch, updated

I landed this and backed it out because it blew up :(.
Attachment #352163 - Attachment is obsolete: true
Attachment #352534 - Flags: checked‑in+
(Assignee)

Updated

9 years ago
Attachment #352532 - Flags: checked‑in+
(Assignee)

Comment 8

9 years ago
Comment on attachment 352532 [details] [diff] [review]
factory.py patch, as landed

This got backed out due to bustage.
(Assignee)

Updated

9 years ago
Attachment #352534 - Flags: checked‑in+
(Assignee)

Comment 9

9 years ago
It looks like mq busted my patch right before I landed it. It doesn't properly handle symlinks. I'll have to try this again another day.
(Assignee)

Updated

9 years ago
Attachment #352532 - Flags: checked‑in+
(Assignee)

Comment 10

9 years ago
Comment on attachment 352532 [details] [diff] [review]
factory.py patch, as landed

This is in for real this time. The buildbot config patch will land tomorrow, so we don't screw up the location of mozconfigs in the meantime.

Checking in factory.py;
/cvsroot/mozilla/tools/buildbotcustom/process/factory.py,v  <--  factory.py
new revision: 1.58; previous revision: 1.57
done
(Assignee)

Comment 11

9 years ago
Comment on attachment 352534 [details] [diff] [review]
buildbot config patch, updated

changeset:   591:6def3b56848b
and
changeset:   593:fe260d5bd948
Attachment #352534 - Flags: checked‑in+
(Assignee)

Comment 12

9 years ago
This change took fine today.
Status: ASSIGNED → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
(Assignee)

Comment 13

9 years ago
The symlinks ended up busting win32. I had to switch them out for real copies for the win32/ mozconfigs.
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.