739486 - test-masters.sh should run ./setup_master.py -t

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: buildbot-configs v1

There is a specific failure mode that ./setup_master.py -t catches that test_master.sh cannot catch at all.  The failure is having a builder name in a scheduler's builder list but not being added to the buildbot master config.  test-masters.sh does not test for this because it only runs multi-master mode, which explicitly skips this buildbot config test.  Setup-master.py -t runs both mutli and single master mode checkconfigs which means that this test is run everywhere.

We've also had setup_master.py -t for quite some time.  I haven't heard of any bugs or major issues with it.  setup_master.py -t is a reimplementation of test-masters.sh in python with enhancements.  Lets change test-masters.sh into a script that runs ./setup_master.py -t instead of doing the test itself.  Leaving the script in place means that we don't need to change preproduction-master or existing workflows.

I just had to debug a failure because of this issue a second time and would really like to not have to do it a third.


An example failure message is:

The check in buildbot is
 811         for s in schedulers:
 812             for b in s.listBuilderNames():
 813                 # Skip checks for builders in multimaster mode
 814                 if not multiMaster:
 815                     assert b in buildernames, \
 816                            "%s uses unknown builder %s" % (s, b)

Comment 1

[Rail Aliiev [:rail]]

FYI, preproduction runs the following instead of test-masters.sh:
python setup-master.py --test --masters-json ../tools/buildfarm/maintenance/production-masters.json

Comment 2

[Chris AtLee [:catlee]]

Comment on attachment 609562 [details] [diff] [review]
buildbot-configs v1

Review of attachment 609562 [details] [diff] [review]:
-----------------------------------------------------------------

lesscode++

Comment 3

[John Ford [:jhford] CET/CEST Berlin Time]

http://hg.mozilla.org/build/buildbot-configs/rev/99c49b20bc91

Comment 4

[Armen [:armenzg]]

These changes went live around 11:55AM PDT.

Comment 5

[[e-mail] (:bhearsum)]

This patch tries to run the wrong script, setup_master.py vs. setup-master.py. I fixed it in this commit: http://hg.mozilla.org/build/buildbot-configs/rev/e5fbe20c3d29