465749 - add http://hg.mozilla.org/releases/mozilla-1.9.1 builds to the production "moz2" unittest master

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

bug 460077 is not going to be ready in time for branching. In the meantime, we'll need unittest builds setup on the existing master. Lukas says we can pull the staging slaves, since unittest in staging is combined with builds and leaks tests. Additionally, we'll need one more mac, and one more linux machine. We can pull these from the production pool.

Comment 1

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Adding Firefox3.1 to production unittest

So here's the initial patch I'm working on.  I don't know how to distinguish to the factory which tinderbox notifier to use.  By having two notifiers, with they both get updated with both sets of builds?

Wanted to use the new unittest factory, but because the a11y tests are enabled in the factory, can't for now.

I also changed the config checkout to using hg clone instead of CVS for both the m-c builds and the firefox3.1

Comment 2

[[e-mail] (:bhearsum)]

Comment on attachment 349085 [details] [diff] [review]
Adding Firefox3.1 to production unittest

>diff -r 6a5a03f10d01 mozilla2-unittest/master.cfg
>--- a/mozilla2-unittest/master.cfg	Wed Nov 19 08:30:46 2008 -0500
>+++ b/mozilla2-unittest/master.cfg	Wed Nov 19 19:59:17 2008 -0500

> c['status'].append(tinderbox.TinderboxMailNotifier(
>                        fromaddr="[e-mail]",
>                        tree="Firefox",
>+                       extraRecipients=["[e-mail]"],
>+                       relayhost="localhost",
>+                       logCompression="bzip2",
>+                       errorparser="unittest"))
>+                       
>+c['status'].append(tinderbox.TinderboxMailNotifier(
>+                       fromaddr="[e-mail]",
>+                       tree="Firefox3.1",
>                        extraRecipients=["[e-mail]"],
>                        relayhost="localhost",
>                        logCompression="bzip2",
>                        errorparser="unittest"))
> 

Please use the 'builders' parameter to limit which ones are sent. You can find examples of this in most of our other Buildbots.


>+c['schedulers'].append(Scheduler(name="firefox3.1",
>+                                 branch="firefox3.1",
>+                                 treeStableTimer=5*60,
>+                                 builderNames=["Linux firefox3.1 moz2-linux-slave10 dep unit test",
>+                                               "Linux firefox3.1 moz2-linux-slave13 dep unit test",
>+                                               "MacOSX firefox3.1 moz2-darwin9-slave05 dep unit test",
>+                                               "MacOSX firefox3.1 bm-xserve22 dep unit test",
>+                                               "WINNT 5.2 firefox3.1 moz2-win32-slave09 dep unit test",
>+                                               "WINNT 5.2 firefox3.1 moz2-win32-slave10 dep unit test"]))

We're going with repository name in the displayed names - please s/firefox3.1/mozilla-1.9.1/



>-moz2_linux_unittest_factory.addStep(ShellCommand, name="download mozconfig",
>-    command=['cvs', '-d', CVSROOT, 'co', '-d', 'mozconfigs',
>-             'mozilla/tools/buildbot-configs/testing/moz2unit/mozconfig-firefox-linux'],
>+moz2_linux_unittest_factory.addStep(ShellCommand, name="clean configs",
>+    command=['rm', '-rf','mozconfigs'],
>+    workdir='.')
>+moz2_linux_unittest_factory.addStep(ShellCommand, name="buildbot configs",
>+    command=['hg', 'clone', self.config_repo_url, 'mozconfigs'],
>     workdir='.')
> moz2_linux_unittest_factory.addStep(ShellCommand, name="copy mozconfig",
>     command=['cp', 'mozconfigs/mozconfig-firefox-linux', 'build/.mozconfig'],
>     workdir='.')

This path is wrong, should be mozilla2/linux-unittest/mozconfig. Please fix it and the ones further down.


>+##
>+# Firefox3.1 Builders
>+##
>+
>+moz2_linux_unittest_31_factory = factory.BuildFactory()
>+moz2_linux_unittest_31_factory.addStep(Mercurial, mode='update',
>+    baseURL='http://hg.mozilla.org/',
>+    defaultBranch='firefox3.1')

This should be 'releases/mozilla-1.9.1'.

>+mozilla2_firefox31_unix_test_builder = {
>+    'name': 'Linux firefox3.1 moz2-linux-slave10 dep unit test',
>+    'slavenames': ['moz2-linux-slave10'],
>+    'builddir': 'trunk_linux-10',
>+    'factory': moz2_linux_unittest_31_factory,
>+    'category': 'HEAD',

'HEAD' is a meaningless term in Mercurial land - just use 'mozilla-1.9.1' or 'firefox-3.1'.



The rest of this looks okay.

Comment 3

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Adding Firefox3.1 to production unittest

With the changes, and also an hgPoller for each branch, and a printChangeset step for each.

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 349199 [details] [diff] [review]
Adding Firefox3.1 to production unittest

>diff -r 6a5a03f10d01 mozilla2-unittest/master.cfg

>+c['status'].append(tinderbox.TinderboxMailNotifier(
>+                       fromaddr="[e-mail]",
>+                       tree="mozilla-1.9.1",

Ah, I forgot to comment on this in the original review - we're using 'Firefox3.1' as the tree here.


>+c['change_source'].append(HgPoller(
>+                 hgURL = "http://hg.mozilla.org/",
>+                 branch = "releases/mozilla-1.9.1",
>+                 pushlogUrlOverride="http://hg.mozilla.org/releases/mozilla-1.9.1/index.cgi/pushlog",

This _might_ be different, but I don't think so. Let's go with this and fix it if we need to.


Please remove the 'rm -rf mozconfigs' step, too, as we talked about.

r=bhearsum with those changes.

Comment 5

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Adding Firefox3.1 to production unittest

With changes.

Comment 6

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Adding Firefox3.1 to production unittest

This is the final patch - ran it in a test buildbot master, config is good, waterfall names now follow consistent naming.

Comment 7

[[e-mail] (:bhearsum)]

Comment on attachment 349226 [details] [diff] [review]
Adding Firefox3.1 to production unittest

changeset:   525:ef80bdf99aa0

Comment 8

[[e-mail] (:bhearsum)]

Okay, these have been enabled on the production unittest master.

I had trouble getting the win32 slaves to work when Buildbot was started through cmd.exe. I had create .bash_profile with the following and start Buildbot from MSYS to make it work:
export PATH=$PATH:/d/mercurial

I suspect it has something to do with the fact that we're running an older MozillaBuild on moz2-win32-slave09 and 10 - as opposed to an updated version on the other win32 unittest slaves.

Test runs are still going on windows, hopefully they'll pass.

Comment 9

[[e-mail] (:bhearsum)]

Alright, both windows machines are up and running now, as is one mac and one linux machine. xserve22 is waiting to be re-imaged (Lukas, did you file a bug for that?), and I'm having trouble with reftest on moz2-linux-slave10. Lukas, when you come around I could use your help there.

This is progressing nicely overall.

Comment 10

[[e-mail] (:bhearsum)]

Of course, right after I say this moz2-darwin9-slave05 vanishes - looking into it.

Comment 11

[[e-mail] (:bhearsum)]

catlee fixed the linux machine - it turns out I set DISPLAY wrong when I started Buildbot.

Comment 12

[[e-mail] (:bhearsum)]

Looks like bm-xserve22 is failing a few reftests. Lukas, do you know if these are the same ones which were failing in staging?

Comment 13

[Lukas Blakk [:lsblakk] use ?needinfo]

It was failing mochichrome and browserchrome iirc, not sure about reftest.

will have to keep an eye on it and see if this is a consistently failing test.

Comment 14

[Lukas Blakk [:lsblakk] use ?needinfo]

Fonts were not present on bm-xserve22 and once copied over from moz2-darwin9-slave05 (after doing a diff on /Library/Fonts between the two) this test passed.


All unittests are green for all 1.9.1 now

Closing.