1653173 - Block release-beetmover-push-to-release on update-verify

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, enhancement)

Description

[Nick Thomas [:nthomas] (UTC+12)]

RyanVM mentioned on Matrix that he's worried about the downsides of pushing to CDNs and then detecting problems in UV. I'm a bit uneasy about this too, but note we have been doing this for automated betas for a while. In case of issues, we ask CloudOps to purge firefox/releases/<version> from archive.m.o and CDNs.

There's history here - in bug 1433461 we started blocking on update verify (Feb. 2018). A month later we'd changed our minds in bug 1446160, at the same time as making any differences change the final state of the task.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

RyanVM, any further thoughts here ? Or opinions from others in Release Management ?

Comment 2

[Ryan VanderMeulen [:RyanVM]]

My impression is that the rationale given in bug 1446160 (not wanting to block releases on known-OK differences) is no longer the case. I haven't seen a failure like those in awhile, so I'm assuming the UV job has gotten smarter over time about what differences to expect and to not complain about.

Given that, I don't see a lot of upside to these jobs being non-blocking other than the time savings for getting the builds into the hands of QA faster. And given the timeout-prone nature of the UV jobs still, that's maybe a strong enough reason for leaving things as-is for now until their reliability is better.

Comment 3

[Tom Prince [:tomprince]]

Yeah, we fixed the known-ok differences in Bug 1461490.

other than the time savings for getting the builds into the hands of QA faster.

If we block push-to-release on update-verify, the notify-promote job will still run as soon as everything is on archive.m.o, even before update-verify is done, so depending on what QA is using for a signal, this won't affect when they get access to builds.

Comment 4

[Tom Prince [:tomprince]]

Attachment: Bug 1653173: [release] Block push-to-release on update-verify being complete; r?RyanVM

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d60e98d8b00d
[release] Block push-to-release on update-verify being complete; r=RyanVM

Comment 6

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/d60e98d8b00d

Comment 7

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/releases/mozilla-beta/rev/1db633bd3dda