486454 - [hg-automation] Move update verify bumping to Updates step

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Nick Thomas [:nthomas] (UTC+12)]

We bump each platform's update verify config separately, which can cause multiple heads in build/tools if they all push back to the repo at the same time. I think we should just bump the verify config after we bump the patcher config, even if that's not quite right logically, because it avoids the heads and we have much the information needed already.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

I'll get this in a bit.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: buildbotcustom part, rearrange factory.py

The key part of this is gutting UpdateVerifyFactory and putting most of it into ReleaseUpdatesFactory, so that we bump the patcher config, bump the verify configs, then create the updates. 

Other changes as part of this
* modify getCandidatesDir() to give universal paths, all our machines have appropriate symlinks at /pub/mozilla.org to the real mount location (there may be other places we can start using this)
* rearrange argument order of ReleaseUpdatesFactory, add extra few vars we need
* not use tagged version of the patcher bumper, since we can't do that for the verify bumper. Let me know if you think this is a bad regression
* d/l old-shipped-locales to build/ instead of tools/release/updates
* tinker with step descriptions
* put all three verifyConfig's into one dict, loop over platforms keys (see buildbot-configs patch)
* rejig ReleaseFinalVerification for new verify config dict

Tested in staging run for 3.5rc1 and seems to be working fine.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: buildbot-configs part

All the changes we need to work with the factory.py rejig, and an updated release_config.py in staging.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

If you're wondering why I didn't put the config bumps in a separate factory it's because we don't check the patcher config in on staging, and we can't guarantee getting the same slave for both config and update generation.

Comment 5

[[e-mail] (:bhearsum)]

Comment on attachment 378844 [details] [diff] [review]
buildbotcustom part, rearrange factory.py

Awesome stuff Nick. I'm happy to see getCandidatesDir using the universal paths, finally. Can you file a followup on getting the rest of the automation to use it, too?

Also: It sortof sucks that we're not using a tag for the patcher tools anymore. We should fix this automation to have a tag everywhere (buildbot-configs, buildbotcustom, tools, patcher, et. al), but that's way beyond the scope of this. (Which is a long way of saying "this is fine".) I filed bug 494161 about this.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Bug 494334 for using getCandidatesDir everywhere.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 378845 [details] [diff] [review]
buildbot-configs part

http://hg.mozilla.org/build/buildbot-configs/rev/d7cd3e71df36

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 378844 [details] [diff] [review]
buildbotcustom part, rearrange factory.py

http://hg.mozilla.org/build/buildbotcustom/rev/3901a3e9cd6e

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

production-master reconfig'd.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Bug 496491 for the Bootstrap equivalent of this fix.