Closed Bug 1402015 Opened 7 years ago Closed 7 years ago

Submit re-compressed MAR metadata to Balrog

Categories

(Release Engineering :: Release Automation: Other, enhancement, P1)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rail, Assigned: rail)

References

Details

Attachments

(8 files, 7 obsolete files)

59 bytes, text/x-review-board-request
sfraser
: review+
Details
4.73 KB, patch
sfraser
: review+
Details | Diff | Splinter Review
961 bytes, patch
mozilla
: review+
Details | Diff | Splinter Review
1.54 KB, patch
bhearsum
: review+
Details | Diff | Splinter Review
1.54 KB, patch
mozilla
: review+
Details | Diff | Splinter Review
59 bytes, text/x-review-board-request
sfraser
: review+
Details
5.47 KB, patch
sfraser
: review+
Details | Diff | Splinter Review
55 bytes, text/x-github-pull-request
jlorenzo
: review+
Details | Review
In bug 1395697 we generate and beetmove complete MARs, but do not submit their metadata to balrog.

The existing script https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/funsize-balrog-submitter/scripts/funsize-balrog-submitter.py has some assumptions, so we need something similar but for completes-only. The release blob names also should be different (suffix="-bz2").
Attached patch WIP gecko patch (obsolete) — Splinter Review
Attached patch WIP releasetasks patch (obsolete) — Splinter Review
The patches "should just work", but better to test them first. Probably with other ones.
Attached patch WIP releasetasks patch (obsolete) — Splinter Review
Attachment #8912306 - Attachment is obsolete: true
Attached patch WIP gecko patch (obsolete) — Splinter Review
Attachment #8912305 - Attachment is obsolete: true
These should cover bug 1402020 as well
Attached patch gecko.diff (obsolete) — Splinter Review
Attachment #8912317 - Attachment is obsolete: true
Attached patch releasetasks.diff (obsolete) — Splinter Review
Attachment #8912316 - Attachment is obsolete: true
Attached patch tools.diff (obsolete) — Splinter Review
Hopefully this is all we need. Now I need to test this in staging!
Comment on attachment 8914356 [details]
Bug 1402015 - Override Balrog blob name suffix, MAR filename pattern, Bouncer product name pattern

https://reviewboard.mozilla.org/r/185620/#review190626
Attachment #8914356 - Flags: review?(sfraser) → review+
Comment on attachment 8914356 [details]
Bug 1402015 - Override Balrog blob name suffix, MAR filename pattern, Bouncer product name pattern

https://hg.mozilla.org/build/tools/rev/a2ecd77a1ee12e13521371ac4dcfe02eab7bb62c
Attachment #8914356 - Flags: checked-in+
Attachment #8912887 - Attachment is obsolete: true
Apparently we don't want to override the "from" suffix, because by default we don't want to use the same suffix. We will be generating 2 blobs, with and without suffix, but the latter is preferable. Also we didn't use the suffix in the past.
Attachment #8914496 - Flags: review?(sfraser)
We don't want to None to be converted to "None".
Attachment #8914512 - Flags: review?(aki)
Attachment #8914512 - Flags: review?(aki) → review+
Attachment #8914496 - Flags: review?(sfraser) → review+
Mixins!!!
Attachment #8914895 - Flags: review?(bhearsum)
Attachment #8914895 - Flags: review?(bhearsum) → review+
Attached patch mar_tools.diffSplinter Review
Attachment #8915047 - Flags: review?(aki)
Attachment #8912884 - Attachment is obsolete: true
Attachment #8912885 - Attachment is obsolete: true
Passing --suffix -bz2 is a PITA!!!
Attachment #8915135 - Flags: review?(sfraser)
Attached file PR for releasetask
Attachment #8915139 - Flags: review?(jlorenzo)
Comment on attachment 8915134 [details]
Bug 1402015 - Submit re-compressed MAR metadata to Balrog

https://reviewboard.mozilla.org/r/186384/#review191464
Attachment #8915134 - Flags: review?(sfraser) → review+
Attachment #8915135 - Flags: review?(sfraser) → review+
Comment on attachment 8915047 [details] [diff] [review]
mar_tools.diff

Oops, sorry for the delay!
Attachment #8915047 - Flags: review?(aki) → review+
Comment on attachment 8915139 [details] [review]
PR for releasetask

Sorry for the delay. LGTM
Attachment #8915139 - Flags: review?(jlorenzo) → review+
Keywords: leave-open
Pushed by raliiev@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/70b3fb79bc92
Submit re-compressed MAR metadata to Balrog r=sfraser a=release DONTBUILD
I tried to push this to beta, but the decision task failed: https://tools.taskcluster.net/groups/XugxWvNaRFOf3SR1hJHlsg/tasks/XugxWvNaRFOf3SR1hJHlsg/runs/0/logs/public%2Flogs%2Flive.log

We need to uplift something under taskcluster/ to make it work.
Probably https://hg.mozilla.org/mozilla-central/rev/79bb52091750#l2.394 is missing. Not sure how safe it would be to uplift this though.
Depends on: 1407279
Attachment #8915139 - Flags: checked-in+
Landed/deployed all the things. I'll keep an eye on it tomorrow.
See Also: → 1411981
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Somehow the patch was backed out in https://hg.mozilla.org/mozilla-central/rev/32faa962a605 O_O
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Looks like a bad conflict resolution in https://hg.mozilla.org/mozilla-central/rev/3f30519845d3
Pushed by raliiev@mozilla.com:
https://hg.mozilla.org/mozilla-central/rev/aebba462abec
Submit re-compressed MAR metadata to Balrog. Fix bad merge. r=me a=release DONTBUILD
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: