Closed
Bug 1091668
Opened 10 years ago
Closed 9 years ago
Generate and sign voucher for plugin-container
Categories
(Release Engineering :: General, defect)
Tracking
(firefox37 fixed, firefox38 fixed, firefox39 fixed)
RESOLVED
FIXED
People
(Reporter: catlee, Assigned: mshal)
References
(Blocks 1 open bug)
Details
(Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/775] )
Attachments
(6 files, 2 obsolete files)
8.17 KB,
text/plain
|
Details | |
398.61 KB,
patch
|
ted
:
review+
gerv
:
review+
|
Details | Diff | Splinter Review |
1.05 MB,
patch
|
ted
:
review+
gerv
:
review+
|
Details | Diff | Splinter Review |
9.29 KB,
patch
|
ted
:
review+
gerv
:
review+
|
Details | Diff | Splinter Review |
3.30 KB,
patch
|
ted
:
review+
|
Details | Diff | Splinter Review |
3.93 KB,
patch
|
glandium
:
review+
|
Details | Diff | Splinter Review |
To support EME, we need to generate a voucher file using a script provided to us by Adobe, and then have this voucher signed. I'm not sure if the script should be incorporated into the build system directly, or if the signing server should take care of the voucher generation and signing.
Comment 1•10 years ago
|
||
We discussed this on IRC a bit. The voucher file script seems to just be generating a hash of the text section, we could pretty easily run that as part of stage-package or something like that. The actual signing of the voucher should happen on the signing server, since it already has private keys.
Reporter | ||
Comment 2•10 years ago
|
||
Comment 3•10 years ago
|
||
assigning to me, per mtg with :coop just now. Not yet certain whats involved, still gathering requirements and necessary milestones
Assignee: nobody → bugspam.Callek
Reporter | ||
Comment 4•10 years ago
|
||
The basic flow here is that the build system will create this voucher, perhaps as part of packaging. The build system will then invoke the regular signing script with a new signing format on the voucher. On the signing server side of things we need to implement the new signing format so that we return a PKCS7 signature. Exactly format is TBD, but the command to run will probably look something like this: openssl smime -binary -sign -signer voucher1.pem -in voucher.bin The result then gets sent back to the build system and included in the packaged build.
Updated•10 years ago
|
Whiteboard: [kanban:engops:https://kanbanize.com/ctrl_board/6/641]
Updated•10 years ago
|
Whiteboard: [kanban:engops:https://kanbanize.com/ctrl_board/6/641] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/775] [kanban:engops:https://kanbanize.com/ctrl_board/6/641]
Assignee | ||
Comment 5•10 years ago
|
||
Looking for some general feedback here: 1) Is there a better way to get the python dependencies (bitstring and pyasn1) in the virtualenv without importing the whole packages? 2) I made it as part of installer since the makensis.mk code was similar and also does signing. We could stick it in stage-package or elsewhere though if that makes more sense. 3) From #c4 it sounds like the actual signing will need to be different. I haven't tested that part yet anyway :)
Attachment #8516884 -
Flags: feedback?(ted)
Assignee | ||
Comment 6•10 years ago
|
||
In case you want to try out patch 0002, here's the part that imports all the dependencies and the voucher script. We still need to sort out whether it makes sense to put the voucher script in the tree, or if we need to download it via tooltool or something for licensing reasons (if we do, that might work around my virtualenv question).
Assignee | ||
Comment 7•10 years ago
|
||
cpearce, what do we need to do with the EME voucher after it gets signed? Is the voucher.bin file included anywhere as part of the package / installer / etc?
Flags: needinfo?(cpearce)
Comment 8•10 years ago
|
||
We haven't received an OK from Adobe to check their voucher generator into a public repo, so I'm making this bug moco-confidential until we receive their OK.
Blocks: EME
Group: mozilla-employee-confidential
Comment 9•10 years ago
|
||
(In reply to Michael Shal [:mshal] from comment #7) > cpearce, what do we need to do with the EME voucher after it gets signed? Is > the voucher.bin file included anywhere as part of the package / installer / > etc? We need to stick the voucher in a well defined place, so that the GMP loading code (GMPChild.cpp specifically) can read that file when it starts up. So please just write the voucher to a file which is installed some place where we can read that file at runtime, and I'll do the rest.
Flags: needinfo?(cpearce)
Comment 10•10 years ago
|
||
OK, I looked back over my notes, we're good to check in the voucher script to a public repo. I will follow up with Adobe about what license they want on the code...
Group: mozilla-employee-confidential
Reporter | ||
Comment 11•10 years ago
|
||
Got more details of how to generate the signature in the correct format. The proper cmdline is: openssl smime -sign -in voucher.bin -signer signer.pem -md sha256 -binary -nodetach -outform DER -out voucher.sig you can verify the signatures with openssl smime -verify -in voucher.sig -inform DER you may need to add -noverify there if you're testing with self-signed certificates
Comment 12•10 years ago
|
||
Mike is working on the build system parts, which align best with this bugs comments thus far, and has patches, I'll spin a new bug for signing-server support.
Assignee: bugspam.Callek → mshal
Comment 13•10 years ago
|
||
This should probably be a Build Config bug but I don't know if that will mess with your kanban stuff.
Comment 14•10 years ago
|
||
Comment on attachment 8516884 [details] [diff] [review] 0002-Make-sign-EME-voucher.patch Review of attachment 8516884 [details] [diff] [review]: ----------------------------------------------------------------- I don't see any simpler way around including the Python dependencies in-tree, it's not that big of a deal. Do we know if this is going to be Windows-only? I think this ought to happen as part of package staging via packager.mk, not just in the installer build. Otherwise this general approach seems fine. ::: toolkit/mozapps/installer/windows/eme/makeeme.mk @@ +6,5 @@ > + > +installer:: > + $(PYTHON) $(topsrcdir)/python/eme/gen-eme-voucher.py -input $(DIST)/bin/plugin-container.exe -output $(DIST)/bin/voucher.bin > +ifdef MOZ_EXTERNAL_SIGNING_FORMAT > + $(MOZ_SIGN_CMD) $(foreach f,$(MOZ_EXTERNAL_SIGNING_FORMAT),-f $(f)) "$(DIST)/bin/voucher.bin" This is probably not what we want, you'll probably want a specific signing format here.
Attachment #8516884 -
Flags: feedback?(ted)
Comment 15•10 years ago
|
||
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #14) > Do we know if this is going to be Windows-only? We'll need to use a different voucher script on other platforms. Currently the script Adobe gave us works on COFF binaries, and obviously that won't work or whatever binary format other platforms use. I'd expect we'd definitely have to do this for MacOSX, and probably have to do it for Linux too.
Comment 16•10 years ago
|
||
(In reply to Chris Pearce (:cpearce) from comment #15) > I'd expect we'd definitely have to do this for MacOSX, and probably have to > do it for Linux too. Why wouldn't we have to do it for Linux, too?
Comment 17•10 years ago
|
||
(In reply to Henri Sivonen (:hsivonen) from comment #16) > (In reply to Chris Pearce (:cpearce) from comment #15) > > I'd expect we'd definitely have to do this for MacOSX, and probably have to > > do it for Linux too. > > Why wouldn't we have to do it for Linux, too? I don't know how we'll do device binding on Linux. I'm not saying it's impossible, just that I don't know what will be acceptable to Adobe, given that anything on a Linux system can be patched.
Comment 18•10 years ago
|
||
(In reply to Chris Pearce (:cpearce) from comment #10) > OK, I looked back over my notes, we're good to check in the voucher script > to a public repo. I will follow up with Adobe about what license they want > on the code... Joe Steele from Adobe has confirmed via email that we should use the following license header on their voucher script: Copyright 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe permits you to use, modify, and distribute this file in accordance with the terms of the Mozilla Public License, v 2.0 accompanying it. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.
Updated•10 years ago
|
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/775] [kanban:engops:https://kanbanize.com/ctrl_board/6/641] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/775]
Comment 19•10 years ago
|
||
Comment on attachment 8516884 [details] [diff] [review] 0002-Make-sign-EME-voucher.patch Review of attachment 8516884 [details] [diff] [review]: ----------------------------------------------------------------- ::: toolkit/mozapps/installer/windows/eme/makeeme.mk @@ +6,5 @@ > + > +installer:: > + $(PYTHON) $(topsrcdir)/python/eme/gen-eme-voucher.py -input $(DIST)/bin/plugin-container.exe -output $(DIST)/bin/voucher.bin > +ifdef MOZ_EXTERNAL_SIGNING_FORMAT > + $(MOZ_SIGN_CMD) $(foreach f,$(MOZ_EXTERNAL_SIGNING_FORMAT),-f $(f)) "$(DIST)/bin/voucher.bin" fwiw, I'm naming the format "emevoucher" over in Bug 1094551. Open to other suggestions though.
Assignee | ||
Comment 20•10 years ago
|
||
I'm splitting the imports of pysan1/bitstring/voucher script into separate patches for easier legal review. r?ted for the location of pyasn1 and r?gerv for licensing (BSD 2-clause)
Attachment #8516885 -
Attachment is obsolete: true
Attachment #8520885 -
Flags: review?(ted)
Attachment #8520885 -
Flags: review?(gerv)
Assignee | ||
Comment 21•10 years ago
|
||
Import bitstring, MIT licensed.
Attachment #8520886 -
Flags: review?(ted)
Attachment #8520886 -
Flags: review?(gerv)
Assignee | ||
Comment 22•10 years ago
|
||
Import the EME voucher script. Adobe copyright, but MPL licensed per #c18.
Attachment #8520887 -
Flags: review?(ted)
Attachment #8520887 -
Flags: review?(gerv)
Assignee | ||
Comment 23•10 years ago
|
||
The actual hook in stage-package. I haven't been able to test the signing properly yet, and won't until bug 1094551 is fixed. However, I don't expect this patch to change much based on that if it needs to.
Attachment #8516884 -
Attachment is obsolete: true
Attachment #8520891 -
Flags: review?(ted)
Comment 24•10 years ago
|
||
Comment on attachment 8520885 [details] [diff] [review] 0001-Bug-1091668-Import-pyasn1-0.1.7.patch Review of attachment 8520885 [details] [diff] [review]: ----------------------------------------------------------------- No problem with this license. Gerv
Attachment #8520885 -
Flags: review?(gerv) → review+
Comment 25•10 years ago
|
||
Comment on attachment 8520887 [details] [diff] [review] 0003-Bug-1091668-Import-eme-voucher-script.patch Review of attachment 8520887 [details] [diff] [review]: ----------------------------------------------------------------- <sigh> What's wrong with the standard MPL 2.0 boilerplate? It says exactly the same thing that Adobe's modified version does. I can imagine asking them to reconsider this won't help this process, so r=gerv, with sadness. Gerv
Attachment #8520887 -
Flags: review?(gerv) → review+
Comment 26•10 years ago
|
||
These bugs are necessary for vouching and sandboxing a third-party CDM.
Blocks: eme-m2
Updated•10 years ago
|
Attachment #8520886 -
Flags: review?(gerv) → review+
Assignee | ||
Comment 27•10 years ago
|
||
FYI try is happy: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=1feb996507ea (The reds are us sorting out some issues on the signing server side). Note we still need to deploy the actual signing keys before this can land, but it's definitely ready for review.
Updated•9 years ago
|
Attachment #8520885 -
Flags: review?(ted) → review+
Updated•9 years ago
|
Attachment #8520886 -
Flags: review?(ted) → review+
Comment 28•9 years ago
|
||
Comment on attachment 8520887 [details] [diff] [review] 0003-Bug-1091668-Import-eme-voucher-script.patch Review of attachment 8520887 [details] [diff] [review]: ----------------------------------------------------------------- It's a little worrying that this is a Python 3 script, in that nothing else we have in-tree is a Python 3 script. Do we have a working python3 binary on the builders?
Updated•9 years ago
|
Attachment #8520891 -
Flags: review?(ted) → review+
Updated•9 years ago
|
Attachment #8520887 -
Flags: review?(ted) → review+
Assignee | ||
Comment 29•9 years ago
|
||
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #28) > It's a little worrying that this is a Python 3 script, in that nothing else > we have in-tree is a Python 3 script. Do we have a working python3 binary on > the builders? This concerned me as well - we don't have python3 on the builders. Fortunately the binary runs fine with python2.7, but since they are using python3, it may mean we get an updated script in the future that requires python3. I'm not sure it makes sense to go ahead and deploy python3 until we actually have a need for it, though.
Comment 30•9 years ago
|
||
Do we need to ask Adobe for a Python 2.7 version of the script, or can we make it work?
Assignee | ||
Comment 31•9 years ago
|
||
(In reply to Chris Pearce (:cpearce) from comment #30) > Do we need to ask Adobe for a Python 2.7 version of the script, or can we > make it work? We execute the script using the $(PYTHON) make variable, which points to a python2.7 installation. This works for us fine now since the script doesn't use any python3 features. The concern would be that since they appear to be using python3 (based on the #!/usr/bin/env python3.3 in the script), they may introduce python3 specific features in the future, which would work fine for them but will require more work for us.
Assignee | ||
Comment 32•9 years ago
|
||
try: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=aceeed1d1bb3 inbound: remote: https://hg.mozilla.org/integration/mozilla-inbound/rev/0536f6db7f66 remote: https://hg.mozilla.org/integration/mozilla-inbound/rev/272903931997 remote: https://hg.mozilla.org/integration/mozilla-inbound/rev/2814563e4abf remote: https://hg.mozilla.org/integration/mozilla-inbound/rev/25800dd79661
Comment 33•9 years ago
|
||
Comment on attachment 8520891 [details] [diff] [review] 0004-Bug-1091668-Make-sign-EME-voucher.patch Review of attachment 8520891 [details] [diff] [review]: ----------------------------------------------------------------- ::: toolkit/mozapps/installer/make-eme.mk @@ +5,5 @@ > +include $(MOZILLA_DIR)/toolkit/mozapps/installer/signing.mk > + > +ifdef MOZ_SIGN_CMD > + ifeq ($(OS_ARCH),WINNT) > + MAKE_SIGN_EME_VOUCHER := $(PYTHON) $(topsrcdir)/python/eme/gen-eme-voucher.py -input $(DIST)/bin/plugin-container.exe -output $(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)/voucher.bin && \ Notwithstanding the fact that I hate that we grow the makefiles involved in packaging, this should be acting on the staged file, not the one in DIST/bin.
Comment 34•9 years ago
|
||
(In reply to Mike Hommey [:glandium] from comment #33) > Comment on attachment 8520891 [details] [diff] [review] > 0004-Bug-1091668-Make-sign-EME-voucher.patch > > Review of attachment 8520891 [details] [diff] [review]: > ----------------------------------------------------------------- > > ::: toolkit/mozapps/installer/make-eme.mk > > + MAKE_SIGN_EME_VOUCHER := $(PYTHON) $(topsrcdir)/python/eme/gen-eme-voucher.py -input $(DIST)/bin/plugin-container.exe -output $(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)/voucher.bin && \ > > Notwithstanding the fact that I hate that we grow the makefiles involved in > packaging, this should be acting on the staged file, not the one in DIST/bin. We explicitly wanted it to act on the post-signed plugin-container.exe.
Comment 35•9 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/0536f6db7f66 https://hg.mozilla.org/mozilla-central/rev/272903931997 https://hg.mozilla.org/mozilla-central/rev/2814563e4abf https://hg.mozilla.org/mozilla-central/rev/25800dd79661
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Comment 36•9 years ago
|
||
(In reply to Michael Shal [:mshal] from comment #29) > This concerned me as well - we don't have python3 on the builders. > Fortunately the binary runs fine with python2.7, but since they are using > python3, it may mean we get an updated script in the future that requires > python3. I'm not sure it makes sense to go ahead and deploy python3 until we > actually have a need for it, though. For clarity, could you change the shebang line so as not to be confusing? If we get an update to the script in the future we'll just have to verify that it continues to work with Python 2.7.
Assignee | ||
Comment 37•9 years ago
|
||
Reopening to address #c33 and #c36
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 38•9 years ago
|
||
Feel free to land a shebang change with rs=me.
Assignee | ||
Comment 39•9 years ago
|
||
Were you looking to generate the voucher from the package-staged or installer-staged file? (dist/firefox/plugin-container.exe or installer-stage/core/plugin-container.exe?) When I tested this before, the voucher generated is the same whether the underlying plugin-container.exe is signed or not. I don't mind updating it either way, but I don't think it will change the result.
Flags: needinfo?(mh+mozilla)
Flags: needinfo?(bugspam.Callek)
Assignee | ||
Comment 40•9 years ago
|
||
I think this is what you're looking for? It should generate the voucher on the signed plugin-container.exe. I also changed the #! python per ted's request, and changed $(topsrcdir) to $(MOZILLA_DIR) for bug 1104997 (if that lands first I'll just rebase). try: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=145633270439
Attachment #8529285 -
Flags: review?(mh+mozilla)
Comment 41•9 years ago
|
||
I would like to write a gtest to test that we can retrieve the generated voucher at runtime, however the voucher is not present in $objdir/dist/bin once `./mach build` has completed locally, so if I wrote a test it would fail locally. Is there a way to run the voucher generator locally and install the (unsigned) voucher in $objdir/dist/bin so that I can have the test past locally at least too?
Comment 42•9 years ago
|
||
Comment on attachment 8529285 [details] [diff] [review] 0001-Bug-1091668-generate-voucher-on-signed-plugin-contai.patch Review of attachment 8529285 [details] [diff] [review]: ----------------------------------------------------------------- ::: toolkit/mozapps/installer/make-eme.mk @@ +6,5 @@ > > ifdef MOZ_SIGN_CMD > ifeq ($(OS_ARCH),WINNT) > + # Note: CWD is DIST > + MAKE_SIGN_EME_VOUCHER := $(PYTHON) $(MOZILLA_DIR)/python/eme/gen-eme-voucher.py -input $(STAGEPATH)$(MOZ_PKG_DIR)/plugin-container.exe -output $(STAGEPATH)$(MOZ_PKG_DIR)/voucher.bin && \ IS it possible to call it plugin-container.voucher to match the naming convention used by the GMP vouchers? Thanks.
Comment 43•9 years ago
|
||
(In reply to Chris Pearce (:cpearce) from comment #42) > IS it possible to call it plugin-container.voucher to match the naming > convention used by the GMP vouchers? Thanks. From a releng-signing-server perspective there is no problem with this. I *will* need to update the signing server config to allow the new file extension as "ok" though, if we go that route.
Comment 44•9 years ago
|
||
If it's not a big hassle, then yes please.
Comment 45•9 years ago
|
||
Comment on attachment 8529285 [details] [diff] [review] 0001-Bug-1091668-generate-voucher-on-signed-plugin-contai.patch Review of attachment 8529285 [details] [diff] [review]: ----------------------------------------------------------------- ::: python/eme/gen-eme-voucher.py @@ +1,1 @@ > +#!/usr/bin/env python why not python2.7?
Attachment #8529285 -
Flags: review?(mh+mozilla) → review+
Updated•9 years ago
|
Flags: needinfo?(mh+mozilla)
Comment 46•9 years ago
|
||
(In reply to Chris Pearce (:cpearce) from comment #41) > I would like to write a gtest to test that we can retrieve the generated > voucher at runtime, however the voucher is not present in $objdir/dist/bin > once `./mach build` has completed locally, so if I wrote a test it would > fail locally. Is there a way to run the voucher generator locally and > install the (unsigned) voucher in $objdir/dist/bin so that I can have the > test past locally at least too? gtests are due to move out of make check in a separate test suite. Some day. It doesn't strike me as problematic to change gtest to run on a packaged build (that is, after make package). Alternatively, you could run an xpcshell test or a mochitest.
Comment 47•9 years ago
|
||
(In reply to Justin Wood (:Callek) from comment #43) > (In reply to Chris Pearce (:cpearce) from comment #42) > > IS it possible to call it plugin-container.voucher to match the naming > > convention used by the GMP vouchers? Thanks. > > From a releng-signing-server perspective there is no problem with this. I > *will* need to update the signing server config to allow the new file > extension as "ok" though, if we go that route. Actually, whatever, we can run with the existing voucher.bin name. It doesn't really matter at this stage what the file is called.
Assignee | ||
Comment 48•9 years ago
|
||
(In reply to Mike Hommey [:glandium] from comment #45) > @@ +1,1 @@ > > +#!/usr/bin/env python > > why not python2.7? Is there a reason to do that? I just used #!/usr/bin/env python because that matches most of the rest of the tree (~660 files), though a good chunk (~115) use #!/usr/bin/python instead. I don't see any that call out python2.7 specifically.
Comment 49•9 years ago
|
||
Seamonkey windows builders have both python2.7 and python, and python is 2.6, not 2.7. Not that I think it matters much for this specific script.
Assignee | ||
Comment 50•9 years ago
|
||
inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/0db3ba4b70ab try: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=52a5df49cbfc
Assignee | ||
Updated•9 years ago
|
Flags: needinfo?(bugspam.Callek)
Comment 51•9 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/0db3ba4b70ab
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Resolution: --- → FIXED
Comment 52•9 years ago
|
||
This regressed somehow; I filed bug 1123990.
Updated•9 years ago
|
Updated•6 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•