656149 - All but pm03 took 2 reconfigs today to pick up the UX branch

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Lukas Blakk [:lsblakk] use ?needinfo]

2011-05-10 07:29:57-0700 [-] configuration update started
2011-05-10 07:30:00-0700 [-] configuration update complete
2011-05-10 15:38:49-0700 [-] loading configuration from /builds/buildbot/build_master6/master/master.cfg
2011-05-10 15:38:54-0700 [-] configuration update started
2011-05-10 15:38:55-0700 [-] configuration update complete

After this morning's early EST reconfig only pm01 had the ux branch live on it even though all the others were using the right configs and showed that they had completed updates. I went around to pm03, bm1 and bm5 and did a second reconfig after which all of them had the ux branch live.

Look into whether there was an import order change or something else - weird that this did not affect pm01.

Comment 1

[Lukas Blakk [:lsblakk] use ?needinfo]

Is this related to a slave list change on this reconfig? That's a known issue that can cause a double reconfig to be needed.

Comment 2

[Lukas Blakk [:lsblakk] use ?needinfo]

Just checked on the mozilla-inbound branch that went into yesterday's reconfig - will confirm with Aki if he did more than one reconfig on any master because the branch is live on all the masters so perhaps this somehow resolved itself or was a glitch from the particulars of the reconfig when UX landed?

Comment 3

[Aki Sasaki (not active)]

Slave change, yes.

Comment 4

[Lukas Blakk [:lsblakk] use ?needinfo]

Need to do testing on this fix but it looks like the issue here is that in mozilla/build_master.cfg and mozilla/scheduler_master.cfg we import master_localconfig _before_ config (which is where the project_branch module is loaded) the dance of the reloads needs to be tweaked to be like mozilla-tests/ where we import config before master_localconfig in both tests_master.cfg and scheduler_master.cfg

Comment 5

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [tested] re-ordering reloads for picking up branches on a reconfig

Comment 6

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 546256 [details] [diff] [review]
[tested] re-ordering reloads for picking up branches on a reconfig

landed on default http://hg.mozilla.org/build/buildbot-configs/rev/4cd95167c302

Comment 7

[Lukas Blakk [:lsblakk] use ?needinfo]

Resolving this, re-open if next new project branch doesn't come up (on build masters) after 1 reconfig.

Comment 8

[Armen [:armenzg]]

Landed on production and reconfigured.