better way to shut off/on nightly updates

NEW
Unassigned

Status

Release Engineering
Balrog: Backend
P3
normal
6 months ago
6 months ago

People

(Reporter: bhearsum, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 months ago
It's relatively common to have the latest Nightly be in a bad state (eg: start up crash or major performance issues), and for us to want to temporarily disable updates to it. Right now the process for doing so is to poke someone with Balrog access (aka RelEng), and hope that they know how to do so. This process works OK as long as someone who knows enough about Balrog to do it is around, which isn't always the case.

This is a common enough case that we should probably find a more streamlined way of handling this. Depending which direction we go, the general "emergency shut off" in bug 1307169 might work. Catlee and I also talked about this recently and came up with the following process:
* Lock live channel to last good release
* After the bad nightlies stop submitting, copy the last known good release to the -latest blob, and point test channel rules at it
* Retriggered nightlies will submit to the -latest blob, and we can test on the test channel
* Once everything looks good, switch the live channel back to the -latest blob

One thing that may make this easier is to switch to a $product-$repo-$date-$revision format for blobs, to make it easier for whatever is doing the locking to find the correct previous nightly.
(Reporter)

Updated

6 months ago
See Also: → bug 1307169
(Reporter)

Updated

6 months ago
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.