424488 - Need test infrastructure for testing signed jars

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Boris Zbarsky [:bzbarsky]]

We need this for testing bug 424188, bug 418996, bug 424426.

What's needed here is a way to have a certificate that the mochitest profile trusts (whatever magic needs to happen in terms of profile setup for that to happen), for us to have a way of putting sets of files into a jar signed with that certificate.  At that point, we can probably use subframes pointing to jar:http:// URIs on one of the mochitest hosts that doesn't get privs by default as needed).  For bonus points, a way to also test jar:file://, once we have file:// mochitests, would be nice.

I doubt we can use a "real" cert for the above (for one thing, they have expiration dates), so I suspect we'd need to add a new root to the mochitest profile, right?  And then somehow have a cert signed by that root that either never expires or is generated at build-time with a new date or some such?  rrelyea, kaie, I'd love your input here.

Comment 1

[Kai Engert [:KaiE:]]

I'll be happy to generate:
- a CA certificate that you can use for testing purposes, 
  which has a lifetime of, say, 25 years
- a "personal" cert, issued by that CA that may be used for object signing,
  with a similar lifetime
- a certificate database which has the above CA trusted for 
  object signing
- a list of commands I used to generate these certs

Using the above, you should be able to check in a static copy of the certificate database and use it for testing (copy cert db into profile directory before starting the test).

If you'd prefer to do build time generation of certs, the list of commands should enable you to implement a script to do it. However, you'd have to build the NSS command line tools in a separate step, because they are currently not being built as part of the mozilla build, and I believe you must build them in their own tree.

Therefore, I propose to start with the CA cert and the personal cert. Please let me know if that sounds sufficient, and I'll go ahead and do it.

Comment 2

[Boris Zbarsky [:bzbarsky]]

Just having the certs and certificate database sounds great.  That should be pretty straightforward to hook up, I think.  Thank you, Kai!

Comment 3

[Kai Engert [:KaiE:]]

mkdir boris
cd boris
certutil -d . -N
(press enter twice for empty password)
(if you're interested, to read certutil help page use: "certutil -H 2>&1 | less" )
certutil -d . -S -n "nonsense-test-ca" -s "CN=Nonsense Test CA, O=Test Org, L=Test Loc, ST=Test State, C=US" -t C,C,C -x -m 0 -v 300 -2 -5
(type chars as prompted, press enter)
(following questions are for -2 argument)
answer "y" to "is this a ca?"
enter to skip path len constraint
answer "n" to "critical extension"
(following questions are for -5 argument)
type 5, enter
type 6, enter
type 7, enter
type 9 to exit, enter
answer "n" to "critical extension"
(use command "certutil -d . -L" to list or "certutil -d . -L -n nonsense-test-ca" to print your shiny new CA cert)
certutil -d . -S -n object-signer -s "CN=Object Signer, O=Test Org, L=Test Loc, ST=Test State, c=US" -c nonsense-test-ca -t p,p,p -m 1 -v 300 -5
(type chars as prompted, press enter)
(following questions are for -5 argument)
type 0, enter
type 1, enter
type 2, enter
type 3, enter
type 9 to exit, enter
answer "n" to "critical extension"
(use command "certutil -d . -L" to list or "certutil -d . -L -n object-signer" to print your shiny new CA cert)


You now have a very powerful certificate database.
It has a CA cert, it has it as trusted, and it even has the private key for that CA.
It also has a trusted object signing cert and its private key.

Usually you'd never find such a db in the wild.
Usually, you'd have 3 variations of dbs:
- a db that has the CA and its private key and keep it in a safe place.
- a db that has the trusted CA's cert (no private key) and the object-signer cert (including private key), which you use for signing jars
- a db which has the trusted CA's cert, only, which is able to verify the signature created with the object-signer cert

Using this single db in all places might work.
However, if you want to do strict testing of, you might want to use the last of the 3 combinations in your test profiles.

Producing that one is easy, you can simply copy the db to a new directory and delete the object-signer cert using:
certutil -d . -D -n object-signer

If you need the object-signer cert in a .p12 file (including private key) you could import into another profile, simply backup it interactively using firefox and cert manager.

If you need the CA cert separately for importing it into another profile (excluding private key), you can simply view the cert in cert manager, click details, and export it to a PEM file.

Comment 4

[Kai Engert [:KaiE:]]

Attachment: certdb-bug424488-ca-and-signer.zip

Comment 5

[Kai Engert [:KaiE:]]

Attachment: certdb-bug424488-ca-only.zip

Comment 6

[Mike Beltzner [:beltzner, not reading bugmail]]

When this gets fixed, we'll want to add a test for bug 427841; dunno how we want to track that sort of thing.

Comment 7

[Boris Zbarsky [:bzbarsky]]

I've been marking the bugs whose tests depend on this as depending on this bug.

Comment 8

[(not currently active) Ted Mielczarek]

This should be pretty easy, as we generate and use a bunch of certs in the Mochitest profile nowadays for use with ssltunnel. Honza Bombas could probably tell you exactly what to do to get this cert into the testing db.

Comment 9

[Dave Townsend [:mossop]]

Attachment: certificate db

I've made some changes to the certificate database, Kai can you take a quick look at the steps I followed to make sure I haven't done anything wrong. The basic changes are some changes to the subjects with better names that fit in with the other certs we have in the mochitest profile and I had to change the trust bits on the object signing cert since signtool rejected it otherwise:

mkdir certdb
cd certdb
certutil -d . -N
(press enter twice for empty password)
(if you're interested, to read certutil help page use: "certutil -H 2>&1 | less" )
certutil -d . -S -n jar-test-ca -s "CN=Signed JAR Certificate Authority, O=Mozilla Testing, OU=Unit Tests" -t C,C,C -x -m 0 -v 300 -2 -5
(type chars as prompted, press enter)
(following questions are for -2 argument)
answer "y" to "is this a ca?"
enter to skip path len constraint
answer "n" to "critical extension"
(following questions are for -5 argument)
type 5, enter
type 6, enter
type 7, enter
type 9 to exit, enter
answer "n" to "critical extension"
(use command "certutil -d . -L" to list or "certutil -d . -L -n jar-test-ca" to print your shiny new CA cert)
certutil -d . -S -n object-signer -s "CN=Object Signer, O=Mozilla Testing, OU=Unit Tests" -c jar-test-ca -t P,P,P -m 1 -v 300 -5
(type chars as prompted, press enter)
(following questions are for -5 argument)
type 0, enter
type 1, enter
type 2, enter
type 3, enter
type 9 to exit, enter
answer "n" to "critical extension"

Comment 10

[Dave Townsend [:mossop]]

Attachment: patch rev 1

This is the actual patch to add the test CA to the mochitest profile. It is mostly as simple as exporting the CA from the DB and adding it to the tree using the following command:

certutil -d . -L -n jar-test-ca -a -o jartests-object.ca

The code that sets up the mochitest profile however needs to grant the object signing CA bits to this CA which it normally doesn't do, ted suggested using a simple flag in the filename to mark that a CA needs this.

I've verified that with this change an XPI signed using the object signing cert in the certificate database installs and validates correctly.

Comment 11

[Dave Townsend [:mossop]]

We might consider checking the certificate database into the tree, but I'm not sure that is really necessary so long as it is always available in this bug.

Comment 12

[Dave Townsend [:mossop]]

Attachment: signed xpi

For completeness this is a signed extension that will install in the mochitest setup.

Comment 13

[Dave Townsend [:mossop]]

Attachment: patch rev 1

un-bitrotted from bug 470963

Comment 14

[Kai Engert [:KaiE:]]

Comment on attachment 357148 [details]
certificate db

I compared my proposed steps from comment 3 and your steps from comment 9, and I see the only differences are the names used for the cert. Looks good to me.
r=kaie

Comment 15

[Kai Engert [:KaiE:]]

After talking with Mossop about a problem he experienced, it looks like you should use "-t ,," when producing the object-signer cert.

Comment 16

[Dave Townsend [:mossop]]

Attachment: certificate db

Updated the certificate DB. Since I just removed the old object signing cert and re-created it the CA cert remains the same and so the patch is still valid. Here are the final set of instructions for completeness:

mkdir certdb
cd certdb
certutil -d . -N
(press enter twice for empty password)
(if you're interested, to read certutil help page use: "certutil -H 2>&1 |
less" )
certutil -d . -S -n jar-test-ca -s "CN=Signed JAR Certificate Authority,
O=Mozilla Testing, OU=Unit Tests" -t C,C,C -x -m 0 -v 300 -2 -5
(type chars as prompted, press enter)
(following questions are for -2 argument)
answer "y" to "is this a ca?"
enter to skip path len constraint
answer "n" to "critical extension"
(following questions are for -5 argument)
type 5, enter
type 6, enter
type 7, enter
type 9 to exit, enter
answer "n" to "critical extension"
(use command "certutil -d . -L" to list or "certutil -d . -L -n jar-test-ca" to
print your shiny new CA cert)
certutil -d . -S -n object-signer -s "CN=Object Signer, O=Mozilla Testing,
OU=Unit Tests" -c jar-test-ca -t ,, -m 1 -v 300 -5
(type chars as prompted, press enter)
(following questions are for -5 argument)
type 0, enter
type 1, enter
type 2, enter
type 3, enter
type 9 to exit, enter
answer "n" to "critical extension"

Comment 17

[Dave Townsend [:mossop]]

Attachment: signed xpi

Updated xpi signed with the new cert

Comment 18

[Dave Townsend [:mossop]]

Comment on attachment 357941 [details] [diff] [review]
patch rev 1

Adds the CA trusted for object signing. Grants that trust using a special flag in the CA filename.

Comment 19

[(not currently active) Ted Mielczarek]

Comment on attachment 357941 [details] [diff] [review]
patch rev 1

+      if root[-7:] == "-object":

if root.endswith("-object"):

Comment 20

[Dave Townsend [:mossop]]

Attachment: for checkin [checked in]

Comment 21

[Dave Townsend [:mossop]]

Landed as http://hg.mozilla.org/mozilla-central/rev/89791850bcd5

Don't know if you want to use this bug for adding actual tests, but all of the mochitest variants will now accept jars signed by the object signer in the certificate db

Comment 22

[Boris Zbarsky [:bzbarsky]]

For what it's worth, the bugs this blocks are where tests should presumably be added.

Comment 23

[Boris Zbarsky [:bzbarsky]]

Oh, and are there docs on how to actually create such a jar, with step-by-step insructions (how to get the cert, how to sign, etc)?

Comment 24

[Dave Townsend [:mossop]]

(In reply to comment #23)
> Oh, and are there docs on how to actually create such a jar, with step-by-step
> insructions (how to get the cert, how to sign, etc)?

https://developer.mozilla.org/en/Signing_a_XPI has some not so great instructions, but roughly:

Download and extract the certificate db from this bug somewhere.
From a command prompt in the directory of files you want in the jar:

signtool -d <path-to-certdb> -k object-signer .
zip signed.jar META-INF/zigbert.rsa
zip -r signed.jar * -x META-INF/zigbert.rsa

Comment 25

[(not currently active) Ted Mielczarek]

(In reply to comment #24)
> Download and extract the certificate db from this bug somewhere.

You should be able to use http://mxr.mozilla.org/mozilla-central/source/build/pgo/certs/, right?

Comment 26

[Dave Townsend [:mossop]]

(In reply to comment #25)
> (In reply to comment #24)
> > Download and extract the certificate db from this bug somewhere.
> 
> You should be able to use
> http://mxr.mozilla.org/mozilla-central/source/build/pgo/certs/, right?

No, the jar CA and object signer are not in that certificate DB

Comment 27

[Dave Townsend [:mossop]]

I'm going to close this bug as fixed since the structure is there to make it possible to write tests for signed jars. The actual development of those tests probably should be done in specific bugs.

Comment 28

[Dave Townsend [:mossop]]

Landed this on the branch: http://hg.mozilla.org/releases/mozilla-1.9.1/rev/e1b21e21f553