825484 - Investigate moving Camino 2.1.x to NSS_3_14_CKBI_1_93_RTM and NSPR_4_9_3_RTM

Status: RESOLVED FIXED

(Camino Graveyard :: General, defect)

Description

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

+++ This bug was initially created as a clone of Bug #685776 +++

Since we need to mess with NSS anyway on CAMINO_2_1_SECBRANCH, it seems like we ought to go ahead and investigate picking up the latest-3.12-branch NSS and certificates (which in turn "requires" NSPR 4.8.9).  Although these obviously haven't been landed on 1.9.2 (and thus haven't been tested with our particular build configuration), they should remain compatible with NSS_3_12_9_RTM and NSPR_4_8_7_RTM.

I've verified (with the following hg commands, to simulate bonsai's old "changes to this directory since foo", and then cross-referencing bugs/checkins/NSPR+NSS release announcements) that NSS 3.12.11 and NSPR 4.8.9 include the local changes landed on 1.9.2 since the last NSPR/NSS updates there:

hg log -l 10 nsprpub/
hg log -l 10 security/coreconf/
hg log -l 10 security/dbm/
hg log -l 20 security/nss/

It should be possible to use the steps in the URL field (plus the note about branches older than Gecko 17) to do this.

Comment 1

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

* Problem #1:

NSS_3_12_11_RTM doesn't contain the latest NSS changes on the NSS_3_12_BRANCH; NSS_3_12_CKBI_1_88_RTM does. Actually, that tag misses the last checkin, bug 711829, which we'd want (it enables some new roots and distrusts some old ones).

So it looks like the proper steps there are 
1) update to NSS_3_12_11_RTM
2) then update to NSS_3_12_CKBI_1_88_RTM
3) then update the certdata files from bug 711829: http://tinyurl.com/bst2bzx

I've checked (http://tinyurl.com/cm9vp6q) and there's only one other fix landed on NSS_3_12_BRANCH post-NSS_3_12_11_RTM that's not in NSS_3_12_CKBI_1_88_RTM or bug 711829; it's a warning fix (bug 686601).

* Problem #2:

Except it looks like 'client.py nss_update' removes everything *except* the changed files in NSS_3_12_CKBI_1_88_RTM when updating to NSS_3_12_CKBI_1_88_RTM :/  So we can't use that to update to NSS_3_12_CKBI_1_88_RTM.

For reference (from mozilla/):

python client.py update_nss NSS_3_12_11_RTM

cd security; rm nss/lib/ckfw/builtins/*; cvs -d :pserver:[e-mail]:/cvsroot export -r NSS_3_12_CKBI_1_88_RTM -d nss mozilla/security/nss/

cd security/nss/lib/ckfw/builtins/; rm certdata.c; cvs -d :pserver:[e-mail]:/cvsroot export -D "2012-01-18" -d builtins mozilla/security/nss/lib/ckfw/builtins/certdata.c

cd security/nss/lib/ckfw/builtins/; rm certdata.txt; cvs -d :pserver:[e-mail]:/cvsroot export -D "2012-01-18" -d builtins mozilla/security/nss/lib/ckfw/builtins/certdata.txt

cd security/nss/lib/ckfw/builtins/; rm nssckbi.h; cvs -d :pserver:[e-mail]:/cvsroot export -D "2012-01-18" -d builtins mozilla/security/nss/lib/ckfw/builtins/nssckbi.h

(This was really the wrong thing to start investigating tonight; I should have just done the other stuff directly :/ )

Comment 2

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Problem #3:

When kaie landed the patch for bug 711829 on the NSS_3_12_BRANCH, he landed the trunk patch, instead of the branch patch (or, more likely, copied the trunk certdata.* files into the branch checkin), since the branch checkin contains the renamed trust flags, which should have been NSS-trunk only.  That appears to break the build.

The solution is to not update the 3 files from CVS, but to apply the branch patch from bug 711829 and then regenerate certdata.c.

After a dozen or so builds and several hours at this, I finally have a build that successfully trusts one of the new roots added in bug 711829 :P

Comment 3

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: Complete diff of upgrade

For reference, here is the complete diff reflecting the following changes:

1) Update NSPR via client.py to NSPR_4_8_9_RTM
2) Update NSS via client.py to NSS_3_12_11_RTM
3) Update NSS via CVS to NSS_3_12_CKBI_1_88_RTM
4) Update NSS via patch to CKBI_1_89
5) Regenerated certdata.c
6) Applied patch from mozilla/security/patches
7) Removed blank lines in prdepend.h and coreconf.dep to force NSS+NSPR rebuilds

Comment 4

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

So for bug 685776, our win was going to be moving to a then-currently-being-updated version of NSS which, although not explicitly tested with our version of Gecko (1.9.0), definitely still supported the same OS combinations (since all Gecko 1.9.x,x<3 had the same requirements, and the version we might upgrade to was in use on 1.9.2).  That was a big win, with the drawback being only the unsure compat with Gecko functions in 1.9.0.

Here, on the other hand, the win is pretty muted; we already have all the cert changes except the final, untagged set (which were clearly never tested on NSS_3_12_BRANCH), the versions of NSS and NSPR, while in theory should be compatible with both our version of Gecko and OS requirements, not only haven't been tested anywhere, but aren't actively being updated.

In other words, the win is basically nil, and the unknowns are huge.  The best win would be simply to apply the patch from bug 711829 directly :P

Another option would be to just update the the current NSS Roots module, which would get us the latest trusted/distrusted certs.  The drawbacks of that include: 

1) trust flags have been renamed (bug 642503 comment 20 and bug 757189)
2) we have no idea what internal changes in NSS might have accompanied trust flag changes without visibility in certdata.*
3) it's obviously untested in our configuration.

(Given that, if we were to do something here, the sanest thing is probably to take every diff beginning at 1.81[1], examine it and revert the trust flags, and then apply.)

[1] http://bonsai.mozilla.org/cvslog.cgi?file=mozilla/security/nss/lib/ckfw/builtins/certdata.txt&rev=HEAD&mark=1.81

Comment 5

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

On the other hand, I was able to build Intel-debug with NSPR_4_9_3_RTM and NSS_3_14_RTM and SSL still works, so that's something…

Comment 6

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

So, after spending an(other) afternoon and evening on this, here's the situation:

1) There isn't enough information (nor do I necessarily know what to do, were we to have the information) to make the old-style knockout certs that NSS 3.12 used.  So

2) We'd have to, at minimum, backport bug 642503 and bug 727204 in order to be able to block bad certs.  That would leave us in the situation that we don't know what the status of NSS is (it's not a tested NSS release, there might be other patches/changes missing), in addition to the other unknowns of not having been tested with our version of Gecko.  I also don't know if those bugs' patches apply remotely cleanly to 3.12, and I just don't have the time these days to chase down the rabbithole on all of that.

OR

We could upgrade to NSS_3_14_CKBI_1_93_RTM (really NSS_3_14_RTM + a patch) and NSPR_4_9_3_RTM.  While it's still not been tested with our version of Gecko, it does still support 10.4+PPC (kaie just checked in a fix for a 10.4-specific bug he found on PPC last month), and it is a tested NSS(+NSPR) release.  Further, it's also currently supported.  Even if we never upgrade NSS+NSPR from that point on, it also provides an up-to-date base to track CKBI changes as needed.

http://www.ardisson.org/smokey/moz/Camino-2.1.3pre-NSS-NSPR-2013-01-03.dmg is a universal build with NSS_3_14_CKBI_1_93_RTM and NSPR_4_9_3_RTM.  

I've done a basic sanity-test on Intel and under Rosetta, and it runs and SSL works.  I'm going to use it as my daily Camino for a couple of days, and I'd appreciate it others would do so as well.

It also has patches for the following bugs, which would be great if people could also test:

* Bug 780390
* Bug 767661

And the sr-expected-final-patches for

* Bug 547485
* Bug 752411

Like I said, there are lots of unknowns with this big of an upgrade, so as much SSL exercise as possible would be great.  Thanks!

Comment 7

[Mark Asbach]

(In reply to Smokey Ardisson (offline for a while; not following bugs - do not email) from comment #6)

> It also has patches for the following bugs, which would be great if people
> could also test:
> 
> * Bug 780390

That's the right-click bug. I can confirm that it is fixed with Camino from the dmg given in comment 6. Tested successfully under Mac OS X 10.8.2 on http://maps.google.com and ChilliProject (for example https://projects.littlestreamsoftware.com/projects/chiliproject_auto_wiki/issues?set_filter=1, right click on issue table cell background). Thanks very much!

Comment 8

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

If no issues with SSL or other unexplained issues in this build come forth by EOD here, I'm going to land this upgrade tonight so that it will be in tomorrow's (8 Jan) nightly.

Comment 9

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: Complete diff of landed upgrade (gzipped)

I'm not sure that anyone cares, but this is the (gzipped, because it's 9+ MB otherwise) diff of the upgrade actually undertaken.

Comment 10

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/b66b287a2650

I'll make the test build update to tomorrow's nightly at some point tomorrow.

Comment 11

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to Smokey Ardisson (offline for a while; not following bugs - do not email) from comment #10)
> http://hg.mozilla.org/releases/mozilla-1.9.2/rev/b66b287a2650
> 
> I'll make the test build update to tomorrow's nightly at some point tomorrow.

Well, I didn't get to that until EOD yesterday, and then I had to refactor my signing script to do the right thing when signing a Camino nightly when switching over from a test build.[1]  But the test build's update-check will now download today's nightly.

(If you want to keep using the test build because of the patches for the other bugs, you should be able to choose either "Remind Me Later" or "Skip This Version"--but if you choose the latter, there's no guarantee I will remember to make the test build update to a future nightly once those other patches have landed.)

[1] http://hg.mozilla.org/users/alqahira_ardisson.org/misc/file/tip/sign_exp.sh