adding a new partner build between `promote` and `push` phase of a release causes failures
Categories
(Release Engineering :: Release Automation, defect, P3)
Tracking
(firefox157 fixed, firefox158 fixed)
People
(Reporter: bhearsum, Assigned: jlorenzo)
References
(Blocks 1 open bug)
Details
Attachments
(4 external files, 2 obsolete files)
After we built Firefox 156.0 we added a new partner to https://github.com/mozilla-partners/repack-manifests/blob/master/default.xml. When the push phase ran for 156.0 it ended up adding builds for this partner, which was surprising and unexpected. These failed because the tasks that were added relied on an earlier task that was cached having certain artifacts in it which it did not.
AFAICT the root problem here is that each time the release promotion action runs, it recalculates the partner list. When this changes between promote and push, the latter ends up creating tasks that don't exist, and therefore can't be re-used through existing_tasks.
Probably the best fix for this is to use a fixed revision for repack-manifests rather than querying with the HEAD of the default branch.
| Assignee | ||
Comment 1•2 months ago
|
||
test_release_promotion registered every artifact fetch by hand as a 303 plus a
200 on the redirect target. Pull that into helpers so the next patch, which adds
three more release promotion tests, does not repeat it five more times.
No behaviour change.
Updated•2 months ago
|
| Assignee | ||
Comment 2•2 months ago
•
|
||
I took a stab at this bug. This is not the fix suggested in the bug. Instead of pinning a repack-manifests
revision, I propose a solution that looks simpler: having the promote action task to fetch the partner config and downstream phases reuse it. Let me know what you think @bhearsum.
| Assignee | ||
Comment 3•1 month ago
|
||
test_release_promotion registered every artifact fetch by hand as a 303 plus a
200 on the redirect target. Pull that into helpers so the next patch, which adds
three more release promotion tests, does not repeat it five more times.
No behaviour change.
Updated•1 month ago
|
| Assignee | ||
Comment 4•1 month ago
|
||
I took a stab at this bug. This is not the fix suggested in the bug. Instead of
pinning a repack-manifests revision, I propose a solution that looks simpler:
having the promote action task to fetch the partner config and downstream phases
reuse it. Let me know what you think @bhearsum.
Updated•1 month ago
|
Comment 5•1 month ago
|
||
firefox-beta Uplift Approval Request
- User impact if declined/Reason for urgency: Bug fix for partner repacks. This patch is only exercised on mozilla-release. If we don't take it, we face the risk of scheduling new and untested partner repacks among the rest.
- Code covered by automated testing?: yes
- Fix verified in Nightly?: no
- Needs manual QE testing?: no
- Steps to reproduce for manual QE testing:
- Risk associated with taking this patch: low
- Explanation of risk level: Partner-repack only. Standard firefox remains unchanged.
- String changes made/needed?: None
- Is Android affected?: no
| Assignee | ||
Comment 6•1 month ago
|
||
test_release_promotion registered every artifact fetch by hand as a 303 plus a
200 on the redirect target. Pull that into helpers so the next patch, which adds
three more release promotion tests, does not repeat it five more times.
No behaviour change.
Original Revision: https://phabricator.services.mozilla.com/D326485
Updated•1 month ago
|
Comment 7•1 month ago
|
||
firefox-release Uplift Approval Request
- User impact if declined/Reason for urgency: Bug fix for partner repacks. This patch is only exercised on mozilla-release. If we don't take it, we face the risk of scheduling new and untested partner repacks among the rest.
- Code covered by automated testing?: yes
- Fix verified in Nightly?: no
- Needs manual QE testing?: no
- Steps to reproduce for manual QE testing:
- Risk associated with taking this patch: low
- Explanation of risk level: Partner-repack only. Standard firefox remains unchanged.
- String changes made/needed?: None
- Is Android affected?: no
| Assignee | ||
Comment 8•1 month ago
|
||
I took a stab at this bug. This is not the fix suggested in the bug. Instead of
pinning a repack-manifests revision, I propose a solution that looks simpler:
having the promote action task to fetch the partner config and downstream phases
reuse it. Let me know what you think @bhearsum.
Original Revision: https://phabricator.services.mozilla.com/D326486
Comment 10•1 month ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/4efc4bdc1ea5
https://hg.mozilla.org/mozilla-central/rev/fe4855f1577f
Comment 11•1 month ago
|
||
| bugherder | ||
Updated•1 month ago
|
Updated•1 month ago
|
Updated•1 month ago
|
Comment 12•1 month ago
|
||
| uplift | ||
Updated•1 month ago
|
Updated•1 month ago
|
Description
•