1318505 - Create tasks to sign Mac builds created in TaskCluster

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task, P1)

Description

[Chris Cooper [:coop] (he/him)]

Mac builds have distinct signing requirements from both Linux and Windows, so we'll need special workers in TC to deal with that, but also so that we have signing covered by the chain-of-trust.

Comment 1

[Chris Cooper [:coop] (he/him)]

(In reply to Chris Cooper [:coop] from comment #0)
> Mac builds have distinct signing requirements from both Linux and Windows,
> so we'll need special workers in TC to deal with that, but also so that we
> have signing covered by the chain-of-trust.

Aki: is this accurate? Have you given any thought to how this might work?

Comment 2

[Aki Sasaki (not active)]

a) signing happens on the signing servers.  Our current signing scriptworkers can talk to the mac signing servers, so they may be able to handle it.
b) if `mach repackage` for macs requires a mac to be able to do packaging, then we may have to spin up mac signing scriptworkers.  However, if we're able to either do packaging on the mac signing servers, or package on linux in a way that is shippable, then we can expand the existing linux signing scriptworkers' capabilities to handle that.

Per my convo with Ben,

ifdef MOZ_SIGN_PREPARED_PACKAGE_CMD  ifeq (Darwin, $(OS_ARCH))    MAKE_PACKAGE    = $(or $(call MAKE_SIGN_EME_VOUCHER,$(STAGEPATH)$(MOZ_PKG_DIR)$(_BINPATH)/$(MOZ_CHILD_PROCESS_NAME).app/Contents/MacOS,$(STAGEPATH)$(MOZ_PKG_DIR)$(_RESPATH)),true) \                      && (cd $(STAGEPATH)$(MOZ_PKG_DIR)$(_RESPATH) && $(CREATE_PRECOMPLETE_CMD)) \                      && cd ./$(PKG_DMG_SOURCE) && $(MOZ_SIGN_PREPARED_PACKAGE_CMD) $(MOZ_MACBUNDLE_NAME) \                      && cd $(PACKAGE_BASE_DIR) && $(INNER_MAKE_PACKAGE)

MOZ_MACBUNDLE_NAME is signed, then dmg-ized.  if we tar that up and publish unsigned, we can do the last 2 steps later.

Since I'm under the impression we can dmg-ize without a mac, we may not need mac signing workers.  We'll flesh out the details with mach repackage for mac.

Comment 3

[Chris Cooper [:coop] (he/him)]

We've been discussing this as part of our migration work week this week. Our constraints were:

* keep the logic for creating packages in-tree
* avoid needing to run configure source checkout on the signing servers in order to sign or package
* break packaging and signing out into distinct steps so they can be retried independently

We've come up with a 3-4 step solution, depending on the platform:

1) Build system generates an unsigned tarball
2) Signing server signs innards: Mac==.app, Windows==.dlls,.exes
3) Package task creates platform-specific package: DMG, zip, exe
4) Sign packaged build (Windows installer only)

This is functionally equivalent to the signing/packaging steps that run inline in the build system right now. 
While we're primarily concerned about Mac in this particular bug, this same sequence is applicable to signing Windows builds.

Comment 4

[Chris Cooper [:coop] (he/him)]

#4 will need to happen for MARs too

Comment 5

[(not currently active) Ted Mielczarek]

FYI, we don't currently do it, but apparently as of OS X 10.11.5 you can now sign DMG files as well, so we should plan for that (even if it doesn't happen in the first pass):
http://stackoverflow.com/a/37923530

Comment 6

[Chris Cooper [:coop] (he/him)]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #5)
> FYI, we don't currently do it, but apparently as of OS X 10.11.5 you can now
> sign DMG files as well, so we should plan for that (even if it doesn't
> happen in the first pass):
> http://stackoverflow.com/a/37923530

Does the DMG signature break non-Sierra platforms or is it ignored?

The way we're setting up signing in TC, we should be able to add in this extra signing task fairly easily if we decide to, but it won't be in the first pass.

Comment 7

[Chris Cooper [:coop] (he/him)]

Kim is looking into this (and dependencies) while Callek is out.

Comment 8

[Kim Moir [:kmoir] ET]

Attachment: IMG_2898.JPG

Comment 9

[Chris Cooper [:coop] (he/him)]

Re-assigning to :Callek. :kmoir will handle the taskgraph bits.

Comment 10

[Kim Moir [:kmoir] ET]

Callek: so since we last spoke re the mac migration work last week some bugs have been reassigned. So to clarify you would work on the build side code of this bug and I would work on the tasks?  It's not clear to me.

Comment 11

[Justin Wood (:Callek)]

(In reply to Kim Moir [:kmoir] from comment #10)
> Callek: so since we last spoke re the mac migration work last week some bugs
> have been reassigned. So to clarify you would work on the build side code of
> this bug and I would work on the tasks?  It's not clear to me.

That is how I understood it.

Comment 12

[Kim Moir [:kmoir] ET]

From irc this morning
aki-away: do you happen to recall re: DMG signing?
9:31 AM 
<coop> Callek: what does the current system do?
9:31 AM IIRC we wanted to keep DMG packaging off the signing servers themselves because it's so intensive
9:32 AM however, if it's easier to have that all contained on the signing server, we can spin up more signing servers
9:32 AM 
<Callek> the current build system, creates a flat directory, that then gets pushed to the signing server as a tarball, and then the build system creates a DMG from what is output
9:33 AM the unsigned taskcluster build creates as an artifact the unsigned DMG though
9:33 AM I vaguely thought we planned to do the unpacking of the DMG on the signing scriptworker, but then do the repacking on the second stage step, but I'm trying to remember what we decided here :/
9:34 AM 
<coop> by "current build system" you mean buildbot-created Mac build?
9:34 AM 
<Callek> yes
9:34 AM 
<coop> do we ever want to sign and not repackage?
9:34 AM 
<Callek> I can't think of a good reason to sign and not then produce a DMG
9:36 AM 
<coop> agreed, but do we then need the DMG logic in more than one place?
9:37 AM we were trying to avoid needing a configured tree on the signing servers IIRC
9:38 AM ⇐ rail quit ([e-mail]) Quit: Gone...
9:40 AM 
<Callek> I think we can get away with this, just trying to get an idea on what is passed back and what will need to be exported to pass in
9:41 AM 
<coop> do we actually need a configured tree to package a DMG?
9:41 AM 
<Callek> I think we probably do, at least if we want to use the build system to do it
9:43 AM 
<coop> ok
9:44 AM so, let me see if i understand where we're at...
9:45 AM signing scriptworker unpacks unsigned dmg from build system, passes contents to signing server for signing, and then repackages signed contents as dmg
9:45 AM accurate?
9:46 AM 
<Callek> That's as I *think* our decision was
9:46 AM Basically, we will have tasks "Build (unsigned)" --> "Signingscriptworker" --> "Mach [re]Package (into deliverable)" ... where the deliverable of unsigned is a .dmg, deliverable of signingscriptworker is a tarball, and deliverable of mach repackage is a dmg again.
9:46 AM aiui
10:06 AM 
<coop> Callek: is that enough to go on, or do you need more detail?
10:10 AM 
<Callek> I can go on that a bit, would love confirmation that I'm not overlooking something, from aki when he gets here though :-)
10:11 AM thanks
10:11 AM 
<coop> let's plan to discuss it at the mtg today
10:13 AM 
<Callek> ok
10:16 AM 
<aki-away> dmg in, tar out is my memory
10:18 AM 3 step for mac, 4 for windows, unless we sign the dmg itself, in which case it would be 4 for both
10:19 AM Callek: ^
10:27 AM 
<Callek> thanks!

Comment 13

[Justin Wood (:Callek)]

Kim is working on this taskgraph part (will file a new bug to track the work + integration of this and other related work)

Comment 14

[Kim Moir [:kmoir] ET]

I'm going to close this.  Bug 1324052 is still open, however, the signing parts have been implemented. I've opened nbug 1352417 (beetmover) and  bug 1352426 (balrog) to address the remaining taskgraph issues.

Comment 15

[Aki Sasaki (not active)]

https://hg.mozilla.org/build/puppet/rev/c6f35f1ab8fa27d035333dc505ceaea893d055a2
bug 1318505 - update signtool to support macapp signing. r=callek