794431 - "ready for release" email can be sent before update verify completes

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, defect)

Description

[[e-mail] (:bhearsum)]

We shouldn't be shipping releases before update verify completes, so we should adjust the "ready for release" scheduler not to fire until after update verify.

Comment 1

[[e-mail] (:bhearsum)]

Looks like this isn't trivial to fix, because the "ready for release" builder gets triggered by the TriggerBouncerCheck scheduler.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

So push to mirrors and deps would have to happen after update verify completed ?

Comment 3

[[e-mail] (:bhearsum)]

(In reply to Nick Thomas [:nthomas] from comment #2)
> So push to mirrors and deps would have to happen after update verify
> completed ?

Definitely not this - don't want to add extra end 2 end time! I was thinking that the "ready for release" builder should be dependent on update verify builders in addition to waiting for necessary uptake.

Comment 4

[[e-mail] (:bhearsum)]

Rail and I poked at this a bit. Almost have it working.

Comment 5

[[e-mail] (:bhearsum)]

Attachment: the patch

Comment 6

[Rail Aliiev [:rail]]

in production

Comment 7

[[e-mail] (:bhearsum)]

Hm, just realized we should block on "final verify" too.

Comment 8

[[e-mail] (:bhearsum)]

Attachment: block on final verify, too

Verified this with a quick local test:
2012-10-10 08:26:13-0400 [-] AggregatingScheduler(release-mozilla-beta-ready-for-release) <id=125340560>: new state: {u'remainingBuilders': [u'release-mozilla-beta-almost_ready_for_release', u'release-mozilla-beta-linux_update_verify_1/4', u'release-mozilla-beta-linux_update_verify_2/4', u'release-mozilla-beta-linux_update_verify_3/4', u'release-mozilla-beta-linux_update_verify_4/4', u'release-mozilla-beta-linux64_update_verify_1/4', u'release-mozilla-beta-linux64_update_verify_2/4', u'release-mozilla-beta-linux64_update_verify_3/4', u'release-mozilla-beta-linux64_update_verify_4/4', u'release-mozilla-beta-macosx64_update_verify_1/4', u'release-mozilla-beta-macosx64_update_verify_2/4', u'release-mozilla-beta-macosx64_update_verify_3/4', u'release-mozilla-beta-macosx64_update_verify_4/4', u'release-mozilla-beta-win32_update_verify_1/4', u'release-mozilla-beta-win32_update_verify_2/4', u'release-mozilla-beta-win32_update_verify_3/4', u'release-mozilla-beta-win32_update_verify_4/4', 'release-mozilla-beta-macosx64_final_verification', 'release-mozilla-beta-win32_final_verification', 'release-mozilla-beta-linux64_final_verification', 'release-mozilla-beta-linux_final_verification'],

u'upstreamBuilders': ['release-mozilla-beta-almost_ready_for_release', 'release-mozilla-beta-linux_update_verify_1/4', 'release-mozilla-beta-linux_update_verify_2/4', 'release-mozilla-beta-linux_update_verify_3/4', 'release-mozilla-beta-linux_update_verify_4/4', 'release-mozilla-beta-linux64_update_verify_1/4', 'release-mozilla-beta-linux64_update_verify_2/4', 'release-mozilla-beta-linux64_update_verify_3/4', 'release-mozilla-beta-linux64_update_verify_4/4', 'release-mozilla-beta-macosx64_update_verify_1/4', 'release-mozilla-beta-macosx64_update_verify_2/4', 'release-mozilla-beta-macosx64_update_verify_3/4', 'release-mozilla-beta-macosx64_update_verify_4/4', 'release-mozilla-beta-win32_update_verify_1/4', 'release-mozilla-beta-win32_update_verify_2/4', 'release-mozilla-beta-win32_update_verify_3/4', 'release-mozilla-beta-win32_update_verify_4/4', 'release-mozilla-beta-macosx64_final_verification', 'release-mozilla-beta-win32_final_verification', 'release-mozilla-beta-linux64_final_verification', 'release-mozilla-beta-linux_final_verification'], u'lastCheck': 1349808299.16667}

Comment 9

[[e-mail] (:bhearsum)]

Now in production.