Closed Bug 2071912 Opened 2 months ago Closed 1 month ago

adding a new partner build between `promote` and `push` phase of a release causes failures

Categories

(Release Engineering :: Release Automation, defect, P3)

Tracking

(firefox157 fixed, firefox158 fixed)

RESOLVED FIXED
Tracking Status
firefox157 --- fixed
firefox158 --- fixed

People

(Reporter: bhearsum, Assigned: jlorenzo)

References

(Blocks 1 open bug)

Details

Attachments

(4 external files, 2 obsolete files)

After we built Firefox 156.0 we added a new partner to https://github.com/mozilla-partners/repack-manifests/blob/master/default.xml. When the push phase ran for 156.0 it ended up adding builds for this partner, which was surprising and unexpected. These failed because the tasks that were added relied on an earlier task that was cached having certain artifacts in it which it did not.

AFAICT the root problem here is that each time the release promotion action runs, it recalculates the partner list. When this changes between promote and push, the latter ends up creating tasks that don't exist, and therefore can't be re-used through existing_tasks.

Probably the best fix for this is to use a fixed revision for repack-manifests rather than querying with the HEAD of the default branch.

See Also: → 1943594
Blocks: 1943594

test_release_promotion registered every artifact fetch by hand as a 303 plus a
200 on the redirect target. Pull that into helpers so the next patch, which adds
three more release promotion tests, does not repeat it five more times.

No behaviour change.

Assignee: nobody → jlorenzo
Status: NEW → ASSIGNED

I took a stab at this bug. This is not the fix suggested in the bug. Instead of pinning a repack-manifests
revision, I propose a solution that looks simpler: having the promote action task to fetch the partner config and downstream phases reuse it. Let me know what you think @bhearsum.

See Also: → 2073107

test_release_promotion registered every artifact fetch by hand as a 303 plus a
200 on the redirect target. Pull that into helpers so the next patch, which adds
three more release promotion tests, does not repeat it five more times.

No behaviour change.

Attachment #9645747 - Flags: approval-mozilla-beta?

I took a stab at this bug. This is not the fix suggested in the bug. Instead of
pinning a repack-manifests revision, I propose a solution that looks simpler:
having the promote action task to fetch the partner config and downstream phases
reuse it. Let me know what you think @bhearsum.

Attachment #9645748 - Flags: approval-mozilla-beta?

firefox-beta Uplift Approval Request

  • User impact if declined/Reason for urgency: Bug fix for partner repacks. This patch is only exercised on mozilla-release. If we don't take it, we face the risk of scheduling new and untested partner repacks among the rest.
  • Code covered by automated testing?: yes
  • Fix verified in Nightly?: no
  • Needs manual QE testing?: no
  • Steps to reproduce for manual QE testing:
  • Risk associated with taking this patch: low
  • Explanation of risk level: Partner-repack only. Standard firefox remains unchanged.
  • String changes made/needed?: None
  • Is Android affected?: no

test_release_promotion registered every artifact fetch by hand as a 303 plus a
200 on the redirect target. Pull that into helpers so the next patch, which adds
three more release promotion tests, does not repeat it five more times.

No behaviour change.

Original Revision: https://phabricator.services.mozilla.com/D326485

Attachment #9645749 - Flags: approval-mozilla-release?

firefox-release Uplift Approval Request

  • User impact if declined/Reason for urgency: Bug fix for partner repacks. This patch is only exercised on mozilla-release. If we don't take it, we face the risk of scheduling new and untested partner repacks among the rest.
  • Code covered by automated testing?: yes
  • Fix verified in Nightly?: no
  • Needs manual QE testing?: no
  • Steps to reproduce for manual QE testing:
  • Risk associated with taking this patch: low
  • Explanation of risk level: Partner-repack only. Standard firefox remains unchanged.
  • String changes made/needed?: None
  • Is Android affected?: no
Attachment #9645750 - Flags: approval-mozilla-release?

I took a stab at this bug. This is not the fix suggested in the bug. Instead of
pinning a repack-manifests revision, I propose a solution that looks simpler:
having the promote action task to fetch the partner config and downstream phases
reuse it. Let me know what you think @bhearsum.

Original Revision: https://phabricator.services.mozilla.com/D326486

Pushed by jlorenzo@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/4cfedcdcea7b https://hg.mozilla.org/integration/autoland/rev/4efc4bdc1ea5 part 1: Extract the release promotion artifact mocks in test_actions.py r=bhearsum,taskgraph-reviewers,releng-reviewers https://github.com/mozilla-firefox/firefox/commit/47da738becf6 https://hg.mozilla.org/integration/autoland/rev/fe4855f1577f part 2: Re-use the partner config resolved by an earlier release phase r=bhearsum,taskgraph-reviewers,releng-reviewers
Status: ASSIGNED → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
Attachment #9645747 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9645748 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9645750 - Attachment is obsolete: true
Attachment #9645750 - Flags: approval-mozilla-release?
Attachment #9645749 - Attachment is obsolete: true
Attachment #9645749 - Flags: approval-mozilla-release?
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: