run update verify on nightlies
Categories
(Release Engineering :: General, enhancement)
Tracking
(Not tracked)
People
(Reporter: bhearsum, Unassigned)
References
Details
(Whiteboard: [updates])
This was originally https://bugzilla.mozilla.org/show_bug.cgi?id=588396, but that bug was closed without the issue actually being fixed.
Essentially, we should run our existing update-verify tests on Nightly. These tests are responsible for verifying that installers and updates get a user to the same place, and typically catch issues where an installer or MAR is missing a file or has one that it shouldn't.
This came up recently after a change to macOS notarization was made which caused us to drop a file accidentally. (See https://github.com/mozilla-releng/scriptworker-scripts/pull/761.)
These tests don't necessarily have to block shipping each Nightly update. Even if they ran after the fact it would be a big benefit, and prevent the issue above from hitting Beta or Release.
As the original bug notes, we could (and maybe should) run these tests in a way that bypasses the need for Balrog. This isn't strictly necessary if we accept that Nightly will ship before update verify runs, though.
|
Reporter | |
Comment 1•9 months ago
|
||
jcristau suggested that we add a new task downstream of repackage (or maybe repackage-signing) that could do the "compare MAR to installer" part of update verify, which is the main part we're concerned about. This would essentially function like a unit test in that it only requires data from the upstream task to run, making it a very tractable solution.
Description
•