412546 - Improve try server wallclock time on win32 & linux by adding slaves

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Each win32 try server job is processed quickly by the win32 slave. However, the job can wait for several hours in a backlog queue of other pending win32 jobs before processing begins. 

To reduce this wallclock time, bhearsum and I discussed adding an additional win32 slave to the try server pool. This is in addition to the existing win32 slave currently being used by the try server. 

After we add this one additional slave, we can monitor turnaround time to see if there's a need for a third slave or not. Also, this seems to be a problem with win32, not mac/linux, but lets watch to see if this extra throughput then shifts the bottleneck to someplace else!

Comment 1

[[e-mail] (:bhearsum)]

Attachment: [checked in] add a second win32 slave

Comment 2

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

From email with Vlad, looks like we need additional linux slaves also, so updating summary.

Comment 3

[[e-mail] (:bhearsum)]

Comment on attachment 297567 [details] [diff] [review]
[checked in] add a second win32 slave

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/tryserver/master.cfg,v  <--  master.cfg
new revision: 1.5; previous revision: 1.4
done

Comment 4

[[e-mail] (:bhearsum)]

try2-win32-slave is up an running. I brought it up in a hurry, so I need to do some password changes and dns fixing tomorrow. I'll start the Linux one cloning tonight.

Comment 5

[[e-mail] (:bhearsum)]

I've got a Linux slave up, waiting on some IT setup in bug 412939

Comment 6

[[e-mail] (:bhearsum)]

The new Linux slave is up and connected. It doesn't have working forward DNS yet however. Waiting on bug 412454 for that.

Comment 7

[[e-mail] (:bhearsum)]

Attachment: [checked in] add a second linux slave

Comment 8

[[e-mail] (:bhearsum)]

Comment on attachment 298481 [details] [diff] [review]
[checked in] add a second linux slave

Checking in master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/tryserver/master.cfg,v  <--  master.cfg
new revision: 1.6; previous revision: 1.5
done

Comment 9

[[e-mail] (:bhearsum)]

Moving try server bugs to the new component.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Ben, the new linux slave needs to have the cvs.m.o fingerprint accepted:

The authenticity of host 'cvs.mozilla.org (10.2.74.107)' can't be established.
RSA key fingerprint is 89:37:28:c1:40:7d:5f:1c:43:12:ed:9e:3c:be:b2:02.
Are you sure you want to continue connecting (yes/no)? 

Comment 11

[[e-mail] (:bhearsum)]

I've accepted the cvs.mozilla.org fingerprint; the machine doesn't appear to have ssh access to build.mozilla.org (hence 412939 being re-opened). I've taken the slave offline until that is taken care of.

Comment 12

[[e-mail] (:bhearsum)]

try2-linux-slave has been connected up to the master now.