Closed Bug 1508056 Opened 2 years ago Closed 2 years ago

Port Bug 1501878 - create a second promote phase graph for deferred mar signing


(Thunderbird :: Build Config, enhancement)

Not set


(Not tracked)

Thunderbird 65.0


(Reporter: rjl, Assigned: rjl)




(2 files)

This is for the Comm port.

+++ This bug was initially created as a clone of Bug #1501878 +++

Currently, this looks like:

- in the partial kind, download target.complete.mar from the repackage kinds instead of repackage-signing kinds.

  - (we don't require that the complete mar is signed to create a partial, and
     if we depend on the signed complete mar, and defer mar signing until signoff,
     we won't generate partials until after signoff.)

  - this was complicated by single_dep dependency crawling. I believe resolving
    bug 1471202 has removed this complication.

- split the repackage-signing into repackage-signing (installer signing) and complete-mar-signing kinds. This will require downstream beetmover and balrog changes.

- allow for another release promotion flavor for the promote graph without mar signing.

for q4. In the future, we'll probably want to:

- add support in ship-it v2 to support the new release promotion flavor
- add a shipitscript task to generate and submit a manifest of unsigned mar files
- add support in ship-it v2 to sign off on the manifest
- add support in signingscript to send the url of the signoffs to autograph along with the signing request
- add support in autograph to require and verify the signoffs for some set of mar signing

I'll probably spawn this second set of tasks out into other bugs and focus on the first set in this one.
Duplicate of this bug: 1508057
Assignee: nobody → rob
From bug 1508057 - There's also a WIP patch in attachment 9025883 [details] [diff] [review].


Running the magic |mach taskgraph ...| locally I get:

ParameterMismatch: missing parameters: signoff_urls, required_signoffs

Grepping through M-C's taskcluster/ci there is no signoff_urls anywhere, and only three mentions of required_signoffs in mar-signing* and partials-signing. I've included the mar-signing* in the patch.
Blocks: 1501878
Pushed by
Port bug 1501878 [create a second promote phase graph for deferred mar signing]. rs=bustage-fix CLOSED TREE
Closed: 2 years ago
Resolution: --- → FIXED
I've landed the patch from bug 1508057, attachment 9025883 [details] [diff] [review], which is pretty much what Rob came up with independently here:
(which also contains some undesired hunks in taskcluster/ci/release-balrog-*).

All that said, the local magic:
mach taskgraph full --root comm/taskcluster/ci -p
failed until a few minutes ago with
  ParameterMismatch: missing parameters: signoff_urls, required_signoffs
Now it works again. Who understands this stuff :-( - I guess Rob, since he said on IRC:

11:31:58 - rjl-: but the reason it didn't work actually had nothing to do with your patch.. it because those signoff_urls and required_signoffs actually have to be added to parameters.yml
11:32:21 - rjl-: which is automagic if a job runs normally, but not so much with the mach taskgraph thing
Error log from a mar-signing task on comm-central.

I suspect there is a misconfiguration in the tb-signing-v1 workers, possibly in the passwords.json file or else in some Taskcluster secrets that I don't have access to.
Will check in with the Taskcluster team.
So should this bug be open until the complete fix is done?
Well, we see the problem on the tree each day, so we won't forget. If no further patches for M-C or C-C are required, it may as well stay resolved since the porting is done as per the summary. In the end, it doesn't matter. Reopen it if you please.
Maybe best to file a separate issue then. Just wanted it to be tracked somewhere
Per Tom, work needs to happen around to add support to prefix format:autograph_hash_only_mar384 with the correct project:comm:thunderbird:releng prefix.
Resolution: FIXED → ---
Is this the bug that's preventing nightlies from getting generated? No nightlies since 2018-11-21
Severity: major → critical
The deferred mar signing feature in Bug 1501878 has a taskcluster scope
value in hardcoded to something Firefox specific. This
patch introduces a new function, get_autograph_format_scope, that will
produce the right value for Firefox and Thunderbird.
Blocks: 1512127
Aki, can the patch on this bug be landed? I assume Rob meant to do so, but he's been away for some time, and meanwhile we haven't had a nightly build for three weeks. :-/
Flags: needinfo?(aki)
Yes, please land.
Flags: needinfo?(aki)
Pushed by
Create function for determining autographing scope. r=aki
Closed: 2 years ago2 years ago
Resolution: --- → FIXED
Aki, Tom, who can approve a beta uplift for

This should have landed at the end of November :-(
Flags: needinfo?(mozilla)
Flags: needinfo?(aki)
You need to log in before you can comment on or make changes to this bug.