412542 - Rename BuildBot buildmasters

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Can we rename the following:

staging-build-console --> staging-1.8-master
build-console --> production-1.8-master
staging-trunk-automation --> staging-1.9-master
production-trunk-automation --> production-1.9-master

This will make it less confusing going forward when we bring up moz2 buildmasters for "trunk". It also makes the various existing buildmasters more consistent.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(obviously, not tied to "master" in name, thats just a suggestion. However, I think the staging/production and the 1.8/1.9 portion really helps clarify things.)

Comment 2

[Robert Helmer [:rhelmer]]

I created CNAMES today so we can start using these names now. Once the VI client is healthy, let's change the vm and hostnames for consistency's sake.

Comment 3

[[e-mail] (:bhearsum)]

I had to make the following change to get production-1.9-master to pickup it's new hostname:
In the zone file:
production-1.9-master IN A 10.2.71.225
production-trunk-automation IN CNAME production-1.9-master

I also changed the hostname listed in dhcpd.conf and in the VI client.

I've made similar changes for the rest of the automation masters. I have to reboot the to get them to pick up their new hostnames, so keeping this bug open for now.

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to comment #3)
> I had to make the following change to get production-1.9-master to pickup it's
> new hostname:
> In the zone file:
> production-1.9-master IN A 10.2.71.225
> production-trunk-automation IN CNAME production-1.9-master
> 
> I also changed the hostname listed in dhcpd.conf and in the VI client.
> 
> I've made similar changes for the rest of the automation masters. I have to
> reboot the to get them to pick up their new hostnames, so keeping this bug open
> for now 

All of the slaves use the old names for the masters, and the bootstrap.cfg does as well. We should fix these up too.

Comment 5

[Robert Helmer [:rhelmer]]

(In reply to comment #4)
> (In reply to comment #3)
> All of the slaves use the old names for the masters

Sorry, to clarify here I just mean the buildbot.tac
We also need to make sure to accept the SSH hostkey, as bootstrap is going to want to ssh from slave->master

Comment 6

[Dave Miller [:justdave]]

FYI, nagios only works with the name that matches the A record for a host.  The nagios on bm-admin01 got broken by the DNS in comment 3 because it's still looking for production-trunk-automation, which is no longer an A record.  I changed it to production-1.9-master in the nagios config and it was happy again.

Comment 7

[[e-mail] (:bhearsum)]

Thanks for catching that.

The other hosts listed in comment #0 have changed too, can you update them as well?

Comment 8

[[e-mail] (:bhearsum)]

Attachment: [checked in] change hostnames in bootstrap configs

This patch shouldn't be checked in until we manually ssh to staging-1.8-master on all of the slaves so we don't hit any unaccepted host key errors.

Comment 9

[[e-mail] (:bhearsum)]

Attachment: change names in buildbot master.cfgs

Comment 10

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Comment on attachment 298322 [details] [diff] [review]
change names in buildbot master.cfgs

Discussed [bots] list on IRC. Also, 'slavename': 'linux-1.8-console1' being changed to 'slavename': 'staging-1.9-master' wont work as linux-1.8-console1 is  a special slave running on same machine as 1.8 master.

Comment 11

[[e-mail] (:bhearsum)]

Attachment: fix staging 1.8 master.cfg

Rob,

John and I were talking about about whether or not the slavename should be the same as the DNS name. I remember having this discussion before, but couldn't find it in my irc logs. Do you remember what we decided on?

Comment 12

[[e-mail] (:bhearsum)]

Attachment: add all bots back into 1.8 configs

Comment 13

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Comment on attachment 298343 [details] [diff] [review]
add all bots back into 1.8 configs

1) bhearsum & I chatted on IRC about using the same identical string (i.e. staging-1.8-master) to refer to two different buildbot processes (one slave, one master) running on the same machine. Its a style difference, which I personally find slightly confusing, but happy to follow bhearsum's preference here.

2) I see 'linux-1.8-console2' now becomes 'production-1.8-master', fixing the previous patch, where 'linux-1.8-console2' became 'production-1.9-master'.

Looks good!

Comment 14

[Robert Helmer [:rhelmer]]

Comment on attachment 298343 [details] [diff] [review]
add all bots back into 1.8 configs

Looks good. I agree that the e.g. "staging-1.8-master" looks weird as a slavename, but it kind of makes sense :P

I think that the real fix is not to run slaves on the master server, which we're moving towards anyway.

Comment 15

[[e-mail] (:bhearsum)]

Attachment: [checked in] patch as landed (had to merge against recent changes)

Checking in production/master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/automation/production/master.cfg,v  <--  master.cfg
new revision: 1.13; previous revision: 1.12
done
Checking in production-1.9/master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/automation/production-1.9/master.cfg,v  <--  master.cfg
new revision: 1.7; previous revision: 1.6
done
Checking in staging/master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/automation/staging/master.cfg,v  <--  master.cfg
new revision: 1.20; previous revision: 1.19
done
Checking in staging-1.9/master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/automation/staging-1.9/master.cfg,v  <--  master.cfg
new revision: 1.17; previous revision: 1.16
done

Comment 16

[[e-mail] (:bhearsum)]

Comment on attachment 298314 [details] [diff] [review]
[checked in] change hostnames in bootstrap configs

Checking in fx-moz18-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz18-bootstrap.cfg,v  <--  fx-moz18-bootstrap.cfg
new revision: 1.28; previous revision: 1.27
done
Checking in fx-moz18-nightly-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz18-nightly-bootstrap.cfg,v  <--  fx-moz18-nightly-bootstrap.cfg
new revision: 1.4; previous revision: 1.3
done
Checking in fx-moz18-staging-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz18-staging-bootstrap.cfg,v  <--  fx-moz18-staging-bootstrap.cfg
new revision: 1.14; previous revision: 1.13
done
Checking in fx-moz19-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz19-bootstrap.cfg,v  <--  fx-moz19-bootstrap.cfg
new revision: 1.9; previous revision: 1.8
done
Checking in fx-moz19-nightly-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz19-nightly-bootstrap.cfg,v  <--  fx-moz19-nightly-bootstrap.cfg
new revision: 1.5; previous revision: 1.4
done
Checking in fx-moz19-staging-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz19-staging-bootstrap.cfg,v  <--  fx-moz19-staging-bootstrap.cfg
new revision: 1.9; previous revision: 1.8
done

Comment 17

[[e-mail] (:bhearsum)]

I'll be updating all of these machines today.

Comment 18

[[e-mail] (:bhearsum)]

I've update the master.cfg and bootstrap.cfgs for production-1.8-master and production-1.9-master. All of the production masters/slaves have got updated buildbot.tac files.

Once the staging machines finish their current run I will be updating those.

Comment 19

[[e-mail] (:bhearsum)]

the staging 1.8 hosts have been updated

Comment 20

[[e-mail] (:bhearsum)]

OK. I've got the staging 1.9 environment set-up now, too. I did a final check on the buildbot.tac files on every machine; nothing is referencing the old masters' names anymore.