418870 - setup talos to test moz2 builds

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[[e-mail] (:bhearsum)]

Comment 1

[[e-mail] (:bhearsum)]

OK.
There's currently mozilla-central + actionmonkey builds for Linux and Win32. All of the win32 builds are busted right now but that shouldn't necessarily stop you from getting the infra up.
mozilla-central builds are here:
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux/
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32/
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/actionmonkey-linux/
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/actionmonkey-win32/

(The last directory doesn't exist yet, because it's never had a working build).

The names of the builders are on the Mozilla2 tinderbox tree (http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla2). I think this is all of the information you'll need.

Comment 2

[alice nodelman [:alice] [:anode]]

Attachment: first attempt at setting up staging actionmonkey/mozilla central linux talos

I'm assuming that there are probably problems in here having to do with the scheduler, since I haven't set one up from scratch before.  The build steps/factories should be fine as they are mostly copies of the existing linux set up.

Comment 3

[[e-mail] (:bhearsum)]

Comment on attachment 306333 [details] [diff] [review]
first attempt at setting up staging actionmonkey/mozilla central linux talos

>Index: master.cfg
>+LINUX_MOZ2_CENTRAL_BUILDER="mozilla-central-linux"
>+LINUX_MOZ2_CENTRAL_BUILDDIR="http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux/"
>+LINUX_MOZ2_ACTIONMONKEY_BUILDER="actionmonkey-linux"
>+LINUX_MOZ2_ACTIONMONKEY_BUILDDIR="http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/actionmonkey-linux/"

Looking at this again, I don't know why I gave you links to ftp.m.o, this should be stage.m.o as it takes time for stage to sync to ftp.


>+                                 "Linux talos moz2 (mozilla central)",
>+                                 "Linux talos moz2 (actionmonkey)",

I think these could just be 'Linux talos mozilla-central' and 'Linux talos actionmonkey'. Doesn't really matter though.


Other than those minor things this looks fine.

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 306333 [details] [diff] [review]
first attempt at setting up staging actionmonkey/mozilla central linux talos

r+ with those changes

Comment 5

[alice nodelman [:alice] [:anode]]

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/master.cfg,v  <--  master.cfg
new revision: 1.40; previous revision: 1.39
done

Comment 6

[alice nodelman [:alice] [:anode]]

Attachment: staging configuration for mac/win moz2 machines

Comment 7

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 310101 [details] [diff] [review]
staging configuration for mac/win moz2 machines

looks good

Comment 8

[alice nodelman [:alice] [:anode]]

Staging configuration for mac/win moz2 machines

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perf-staging/master.cfg,v  <--  master.cfg
new revision: 1.4; previous revision: 1.3
done

Comment 9

[alice nodelman [:alice] [:anode]]

There are now win, mac & linux moz2 testers reporting to MozillaTest.

Comment 10

[alice nodelman [:alice] [:anode]]

Attachment: push mozilla-central talos machines to production

These machines are successfully cycling and reporting to MozillaTest - they can now be moved to the Mozilla2 waterfall.  This patch is for mozilla-central only, actionmonkey will remain in staging.

Comment 11

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 313194 [details] [diff] [review]
push mozilla-central talos machines to production

what's a "MultiBuildScheduler" and where does it get installed?

Comment 12

[[e-mail] (:bhearsum)]

MultiBuildScheduler will trigger a build X number of times instead of only 1. I was hoping it could go into buildbotcustom (this would require a checkout of buildbotcustom on the talos master).

Comment 13

[alice nodelman [:alice] [:anode]]

Push mozilla-central to production:

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/master.cfg,v  <--  master.cfg
new revision: 1.51; previous revision: 1.50
done

Comment 14

[alice nodelman [:alice] [:anode]]

Backed out the patch, wouldn't apply to talos buildbot master with:

          File "/home/buildbot/builds/lib/python2.4/site-packages/buildbot/master.py", line 684, in loadConfig
            for name, passwd in bots:
        exceptions.ValueError: too many values to unpack

Have we hit the upper limit of the amount of bots allowed for a master?  Or is this just some typo in the bots list that I'm not noticing...

Comment 15

[alice nodelman [:alice] [:anode]]

Push mozilla-central to production take 2:

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/master.cfg,v  <--  master.cfg
new revision: 1.53; previous revision: 1.52
done

Comment 16

[Benjamin Smedberg]

This is done for mozilla-central, so removing blocking status... leaving this open for actionmonkey.

Comment 17

[alice nodelman [:alice] [:anode]]

Current status of actionmonkey:

- 3 linux machines testing
- 3 mac tiger machines testing

Blocking on winxp machines because we are currently not successfully generating actionmonkey builds on winxp for testing.

Comment 18

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

updated during triage.

Comment 19

[alice nodelman [:alice] [:anode]]

In the interest of finishing up the talos set up I've temporarily hooked up the win32 actionmonkey testers to mozilla-central builds - this will help assure us that the machines are correctly configured so that once win32 actionmonkey builds become available we'll be ready to go.

Once the configuration has been assured on all platforms these machines can be pushed to production.

Comment 20

[alice nodelman [:alice] [:anode]]

Attachment: push actionmonkey talos machines to production

Even though we are not producing actionmonkey windows builds I've testing the talos configuration on the win talos boxes with mozilla-central builds - so they'll be ready once builds become available.

The numbers for the linux/mac actionmonkey machines are also stable and ready to go.

Comment 21

[alice nodelman [:alice] [:anode]]

Checking in perfmaster/master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/master.cfg,v  <--  master.cfg
new revision: 1.55; previous revision: 1.54
done
Checking in perf-staging/master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perf-staging/master.cfg,v  <--  master.cfg
new revision: 1.11; previous revision: 1.10
done

Comment 22

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

From moz2 meeting this week:

1) It going to take at least a couple of weeks to resolve bug#428529.

2) Its sufficient to take the talos machines setup for actionmonkey, reassign them to mozilla-central, confirm they work ok, and then reassign them back to the broken actionmonkey branch. With those experiments done, people were fine with removing blocking dependency on bug#428529. At that point, we can close this bug#418870.

3) Once the actionmonkey branch works again, the talos machines will see the actionmonkey builds and start testing them. As we know the same machines work on mozilla-central, any differences in behavior will be investigated by the actionmonkey folks.

Comment 23

[alice nodelman [:alice] [:anode]]

All the machines that are currently up are now reporting to the Mozilla2 waterfall.

Comment 24

[alice nodelman [:alice] [:anode]]

Now working on setting up

- mozilla-central vista machines
- mozilla-central leopard machines
- mozilla-central nochrome testers on all platforms

That should pretty much cover all the necessary machines for complete talos coverage of mozilla-central.

Comment 25

[alice nodelman [:alice] [:anode]]

I've split out the nochrome/fast cycle moz2 machines into bug 436368.  This bug covers the primary talos support for moz2 - talos triads on all five platforms.

We'll consider this bug completed when the vista & leopard talos sets are pushed to production.

Comment 26

[alice nodelman [:alice] [:anode]]

Attachment: push mozilla-central vista talos machines to production

Comment 27

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 322991 [details] [diff] [review]
push mozilla-central vista talos machines to production

mysterious extra line in the Status comment section. Otherwise, cromulent.

Comment 28

[alice nodelman [:alice] [:anode]]

Removed mysterious extra line.

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/master.cfg,v  <--  master.cfg
new revision: 1.60; previous revision: 1.59
done

Comment 29

[alice nodelman [:alice] [:anode]]

Attachment: push mozilla-central leopard talos machines to production

Comment 30

[[e-mail] (:bhearsum)]

Comment on attachment 323134 [details] [diff] [review]
push mozilla-central leopard talos machines to production

happy friday!

Comment 31

[alice nodelman [:alice] [:anode]]

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/master.cfg,v 
<--  master.cfg
new revision: 1.61; previous revision: 1.60
done

Comment 32

[alice nodelman [:alice] [:anode]]

Status update:

- fast cycle machine currently reporting to talos stage
- nochrome machines blocked on racking/imaging of newly purchased minis

Comment 33

[alice nodelman [:alice] [:anode]]

- nochrome machines up and reporting.

All done here.