414456 - set up qm-pmac-fast01, qm-plinux-fast01, qm-pxp-fast01

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[alice nodelman [:alice] [:anode]]

From the set of mac mini machines that is currently idle and not imaged I'd like to take out three and configure them as rapidly cycling talos boxes using the current talos machine images.  This will be to fulfill the requirements of bug 413713.

For the mac machine we have to block on bug 414400 (set up 10.4.8 mac image).

Also, if you have better machine name suggestions I'd be happy to hear them.

Comment 1

[Reed Loden [:reed]]

Will these be tier 1 like the current old perf machines? I hope so.

Comment 2

[alice nodelman [:alice] [:anode]]

Let's get them up and to stage, then we can figure out what we're doing with them.

Comment 3

[Justin Fitzhugh]

No, mini's aren't tier 1 as we have built redundancy into the talos sets.  In other words if one goes down in the middle of the night, it won't page someone given we have 2 others doing the same thing.

Comment 4

[Reed Loden [:reed]]

Then they shouldn't be minis if minis can't be tier 1. The machines these are replacing are tier 1 (and for good reason), as they are our first line of testing when checking for perf regressions. Also, just because we have three machines running the same tests doesn't mean that they are redundant for each other, as they are on different platforms, so they aren't the "same thing".

Comment 5

[Justin Fitzhugh]

For the record, bl-bldlnx* are have been explicitly called out as *not* tier one as they are in the landings office.  I'll leave it to Alice to determine the support level of the machines as she understands the usage for them most.  For now we are just staging so we'll go with 3x mini's.

Comment 6

[Reed Loden [:reed]]

(In reply to comment #5)
> For the record, bl-bldlnx* are have been explicitly called out as *not* tier
> one as they are in the landings office.

My understanding is that the bl-bld* machines in the office are considered tier 1 for all software issues but not tier 1 for hardware problems (as there are no remote hands), which seems completely reasonable. If this is not your understanding too, then this needs to be resolved, as these machines definitely cause the tree to close if there is a problem with one of them.

Please note that http://wiki.mozilla.org/Build:Farm#Linux considers bl-bldlnx* as tier 1, as does bug 384966, comment #16 and #17. Bug 404013, comment #20 also mentions this.

Comment 7

[Mike Schroepfer]

We need them to be mini's so we can compare numbers roughly across these + the existing talos boxes.  So either make them tier1 or buy spares if you need to.

Comment 8

[Justin Fitzhugh]

(In reply to comment #6)
> (In reply to comment #5)
> > For the record, bl-bldlnx* are have been explicitly called out as *not* tier
> > one as they are in the landings office.
> 
> My understanding is that the bl-bld* machines in the office are considered tier
> 1 for all software issues but not tier 1 for hardware problems (as there are no
> remote hands), which seems completely reasonable. If this is not your
> understanding too, then this needs to be resolved, as these machines definitely
> cause the tree to close if there is a problem with one of them.
> 
> Please note that http://wiki.mozilla.org/Build:Farm#Linux considers bl-bldlnx*
> as tier 1, as does bug 384966, comment #16 and #17. Bug 404013, comment #20
> also mentions this.
> 

thanks for pointing me at that - just modified the doc to make hardware on
those two boxes tier 2.  software at tier 1 is fine (left that on the doc too).

Comment 9

[Phong Tran [:phong]]

qm-pmac-fast01	OS X 10.4.8	ym7392z5yl1	1304			103.06.502
qm-pxp-fast01	Windows XP	ym7393bayl1	1305	10.2.73.218	103.06.503	
qm-plinux-fast01  Ubuntu	ym7392s4yl1	1306	10.2.73.210	103.06.504

Comment 10

[alice nodelman [:alice] [:anode]]

The mac machine will have to be re-imaged with the fresh one from bug 414400 (once we have resolved the issues in that bug).

Comment 11

[alice nodelman [:alice] [:anode]]

Just to update, all of these machines are now up and reachable and no longer need any imaging.

Comment 12

[alice nodelman [:alice] [:anode]]

Attachment: add fast machines to stage

Initial push of machines to stage.

Comment 13

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 301409 [details] [diff] [review]
add fast machines to stage

you might want to remove the contents of the "win32-trunk-mini-nochome" directories on the master and slaves when the new config takes effect. This patch looks crisp.

Comment 14

[alice nodelman [:alice] [:anode]]

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/master.cfg,v  <--  master.cfg
new revision: 1.33; previous revision: 1.32
done
RCS file: /cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/configs/historic_manifest.txt,v
done
Checking in configs/historic_manifest.txt;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/configs/historic_manifest.txt,v  <--  historic_manifest.txt
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/configs/fast.sample.config,v
done
Checking in configs/fast.sample.config;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/configs/fast.sample.config,v  <--  fast.sample.config
initial revision: 1.1
done

Comment 15

[alice nodelman [:alice] [:anode]]

I've gotten my first results in from qm-pxp-fast01 and qm-plinux-fast01 (there is an ongoing problem with the imaging of mac test boxes that has resulted in the configuration of qm-pmac-fast01 being delayed).  The numbers reported for ts, tdhmtl and twinopen are consistent with other talos machines.

It is looking like a sub-15 minute cycle time for both.  Look at the MozillaTest tinderbox waterfall for the trunk fast machines.

Comment 16

[Justin Fitzhugh]

can this move out of server ops?  think our part is done...

Comment 17

[alice nodelman [:alice] [:anode]]

The mac machine is still blocked on bug 414400, but we can move this bug to core testing.

Comment 18

[alice nodelman [:alice] [:anode]]

qm-pmac-fast01 now up and reporting to MozillaTest.  If the numbers on these three machines look good we should consider pushing to production early next week.

Comment 19

[alice nodelman [:alice] [:anode]]

Attachment: push fast machine to production

Comment 20

[alice nodelman [:alice] [:anode]]

Push to production.

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/master.cfg,v  <--  master.cfg
new revision: 1.35; previous revision: 1.34
done
RCS file: /cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/configs/fast.production.sample.config,v
done
Checking in configs/fast.production.sample.config;
/cvsroot/mozilla/tools/buildbot-configs/testing/talos/perfmaster/configs/fast.production.sample.config,v  <--  fast.production.sample.config
initial revision: 1.1
done

Comment 21

[alice nodelman [:alice] [:anode]]

up and reporting.

Comment 22

[(not currently active) Ted Mielczarek]

Mass move of Core:Testing bugs to mozilla.org:ReleaseEngineering. Filter on RelEngMassMove to ignore.