459699 - stop running certutil at build time, make developers run it when adding new SSL domains to server-locations.txt

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[(not currently active) Ted Mielczarek]

Currently we run genpgocert.py to build a NSS cert database for ssltunnel at build time. This is a problem in cross-compile situations, because we're not building a host binary of certutil. Since server-locations.txt shouldn't change all that often, I think we should push this requirement onto developers instead. We can commit a pre-built cert db to the repository, and just update it when new SSL hosts are added to the list. We can probably reuse the existing scripts, and just add a makefile target somewhere that will read server-locations.txt and rebuild the database.

Comment 1

[Honza Bambas (:mayhemer)]

Attachment: fix v1 [Checkin: Comment 8]

- there is pre-generated certificate database with the CA+private key inside and the server cert+private key inside
- 

Also included server cert+priv key for bug

Comment 2

[Honza Bambas (:mayhemer)]

Comment on attachment 343426 [details] [diff] [review]
fix v1
[Checkin: Comment 8]

hit enter too soon... this is complete comment:

- there is pre-generated certificate database with the CA+private key inside
and the server cert+private key inside
- when server-location.txt is modified do:
 1. run make at /testing/mochitest
 2. run make genservercert at /build/pgo 

Also included server cert+priv key for bug 413909.

Comment 3

[(not currently active) Ted Mielczarek]

Comment on attachment 343426 [details] [diff] [review]
fix v1
[Checkin: Comment 8]

Most of this patch looks fine, but what the heck is up with this:

+def dbFilesExist(path):
+  for root, dirs, files in os.walk(path):
+    for name in files:
+      for dbFile in dbFiles:
+        if dbFile.match(name) and os.path.exists(os.path.join(root, name)):
+          return True
+  return False
+
+def installDbFiles(path, dest):
+  for root, dirs, files in os.walk(path):
+    for name in files:
+      for dbFile in dbFiles:
+        if dbFile.match(name):
+          shutil.copy(os.path.join(root, name), os.path.join(dest, name))

Why are you walking the entire srcdir to find the db files? You know where they are, just pass that path in from the Makefile or hardcode it in the script. This will make the rebuild take way longer than it needs to.

Comment 4

[Honza Bambas (:mayhemer)]

(In reply to comment #3)
> Why are you walking the entire srcdir to find the db files? You know where they
> are, just pass that path in from the Makefile or hardcode it in the script.
> This will make the rebuild take way longer than it needs to.

Oh, I do not walk the whole tree. The parameter 'path' is filled with path to cert dir (build/pgo/certs) in the source tree (as you suggest). I need to walk the dir because I match file names to regular expression (I need to find all cert*.db, key*.db and secmod.db files). And it is just a millisecond on all my machines to execute...

Comment 5

[(not currently active) Ted Mielczarek]

Comment on attachment 343426 [details] [diff] [review]
fix v1
[Checkin: Comment 8]

Ok. That still seems like a little bit of overkill. You should have a known path name and a known set of files, you shouldn't have to invoke |os.walk| to find them. If it's only walking that one directory though, it's not so bad. I'd prefer something simpler, but this is ok.

Comment 6

[Honza Bambas (:mayhemer)]

Perfect would be if os.walk took a mask which files I want to process. I have chosen this approach because there are different versions of the certificate database (cert8.db, cert9.db etc) and we want to handle them all.

Comment 7

[(not currently active) Ted Mielczarek]

Fair enough, this patch is fine as is.

Comment 8

[Serge Gautherie (:sgautherie)]

Comment on attachment 343426 [details] [diff] [review]
fix v1
[Checkin: Comment 8]

http://hg.mozilla.org/mozilla-central/rev/2e2025e49c84