357333 - Branches only: build using a supported NSS 3.11 tag

Status: VERIFIED FIXED

(Core :: Security: PSM, defect)

Description

[Kai Engert (:KaiE:)]

As of today, 1.8 uses the NSS 3.11.3 release plus some selected patches.

In order to synch the 1.8 branch to a supported NSS snapshot, let's make use Mozilla PSM go to NSS_3_11_20060926_TAG, which is the first snapshot after the 3.11.3 release that contains all the patches we need at this time.

I will attached a diff between the current Mozilla 1.8 branch snapshot and the new proposed snapshot:
  cvs diff -u -r MOZILLA_1_8_BRANCH -r NSS_3_11_20060926_TAG 
           mozilla/security/nss mozilla/security/coreconf

That diff will contain a lot of noise, because NSS sources make use of CVS variables like $Id. And it will list differences in areas not built by Mozilla like nss/cmd and nss/tests.

Because of that I will also attach a diff that has the noise removed - for illustration of the real changes.

Finally I will attach the patch that I'm proposing to check in: A change to mozilla/client.mk to pull that tag. This will change the 1.8 branch to pull from the direct NSS source tag (which is guaranteed not to be a moving tag). Switching to that approach will simplify future landing of updated stable security snapshots on the branch - there would be no more checkins to MOZILLA_1_x_BRANCHes for mozilla/security/nss

Comment 1

[Kai Engert (:KaiE:)]

(In reply to comment #0)
>   cvs diff -u -r MOZILLA_1_8_BRANCH -r NSS_3_11_20060926_TAG 
>            mozilla/security/nss mozilla/security/coreconf

cvs rdiff

Comment 2

[Kai Engert (:KaiE:)]

Attachment: full diff, for illustration

Comment 3

[Kai Engert (:KaiE:)]

Attachment: diff with noise removed, for illustration

Comment 4

[Kai Engert (:KaiE:)]

Attachment: Patch v1

Proposed patch for check in to 1.8 branch.

Comment 5

[Kai Engert (:KaiE:)]

Clarifying summary of bug.
This change is intended for the stable Mozilla branches ONLY.
Not for trunk.
(The proposed snapshot had landed on the trunk with bug 354378. By now the trunk already uses a newer snapshot of NSS.)

Comment 6

[Kai Engert (:KaiE:)]

Comment on attachment 242820 [details] [diff] [review]
Patch v1

I'm assuming this is what we want to do for Mozilla 1.8 (FF 2.0.x) and Mozilla 1.8.0 (FF 1.5.0.x).

Can you please review / approve?

Not sure whether this should go into 1.8.0.8 or 1.8.0.9.

Comment 7

[Daniel Veditz [:dveditz]]

Comment on attachment 242820 [details] [diff] [review]
Patch v1

Not 1.8.0.8, we're code complete, wouldn't take this huge change w/out more testing time. Will leave on the table for 1.8.0.9

Comment 8

[Wan-Teh Chang]

Comment on attachment 242820 [details] [diff] [review]
Patch v1

Kai, you can use the -kk option for cvs diff to omit the
changes to RCS keywords.

Nelson should review this patch.  I'm not sure if it's
acceptable to use a non-release snapshot of NSS in a
Firefox release.

Note that the Mozilla 1.8 branch doesn't use the
NSS_CO_TAG to pull mozilla/dbm.  This is not a serious
problem.  In fact, pulling mozilla/dbm by the
NSS_3_11_20060926_TAG may undo the MPL/GPL/LGPL
tri-licensing in mozilla/dbm.

Comment 9

[Wan-Teh Chang]

Comment on attachment 242820 [details] [diff] [review]
Patch v1

Kai, let's wait until we have NSS_3_11_4_RTM to do this.

This patch would only work for the MOZILLA_1_8_BRANCH.
The MOZILLA_1_8_0_BRANCH also needs the patch for bug 317620
because of the new freebl shared libraries added in NSS 3.11.

There is another good reason for the MOZILLA_1_8_0_BRANCH to
upgrade to NSS 3.11.x -- to pick up the NSS crypto module
that is undergoing FIPS 140-2 validation.  But I'm afraid
that backporting the patch for bug 317620 is a lot of work.

Comment 10

[Nelson Bolyard (seldom reads bugmail)]

When diffing two sets of files with different tags in the repository
(as opposed to diffing files in one's work area against the respository)
you can use the -kk option to cvs diff to suppress all diffs in RCS keywords.

Comment 11

[Daniel Veditz [:dveditz]]

How close are you to 3.11.4 rtm? Given we're shipping 2.0.0.1 soon with or without 3.11.4 is it better to take Kai's proposed tag or leave it alone and get 3.11.4 in 2.0.0.2?

We seem to be pretty close to 3.11.3 plus certdata changes and one other bug.

Agreed we need to move 1.8.0 onto a supported NSS 3.11 release, but is now the right time? what more do we need to do beyond updating the tag in the makefile?

Comment 12

[Wan-Teh Chang]

Dan, when will you stop accepting fixes for FF 2.0.0.1?
We need 3-5 days for NSS release QA.

We do have a NSS_3_11_4_BETA1 CVS tag now, but I need to
make sure FF 2.0.0.1 uses an NSS RTM tag.  So if I have
the FF 2.0.0.1 checkin cutoff date, I can do some planning.

Another good reason for FF 2.0.0.x to take NSS 3.11.4 is
that it is our best prediction of what will be FIPS validated.

Comment 13

[Daniel Veditz [:dveditz]]

That'll make it, I think we're shooting for being done by Thanksgiving so we can start builds when we get back.

Does this work for the 1.8.0 branch as well? Do we need to change our NSPR to match?

Comment 14

[Daniel Veditz [:dveditz]]

We're not going to block 1.8.0.9 for this, but we'd like to do it if you think you can pull it off. If not there's always 1.8.0.10 coming down the pike.

Jay says the code freeze is Nov 17, the week before Thanksgiving.

Comment 15

[Wan-Teh Chang]

The MOZILLA_1_8_0_BRANCH needs more work because we need to
modify all the installer/package files to add the freebl3
shared library, which was added in NSS 3.11.

We will try to release NSS 3.11.4 on Nov. 17.  Our release
build and QA process takes three days, so we may miss the
FF 2.0.0.1 code freeze deadline.

NSS 3.11.4 will require NSPR 4.6.4.  The MOZILLA_1_8_BRANCH
is using NSPR 4.6.3 now.  NSPR 4.6.4 has four bug fixes listed in
http://www.mozilla.org/projects/nspr/release-notes/nspr464.html#What%27s

Comment 16

[Wan-Teh Chang]

Attachment: Patch for the MOZILLA_1_8_BRANCH to use NSPR 4.6.4 and NSS 3.11.4

We released NSS 3.11.4 and NSPR 4.6.4 last Friday evening (2006-11-17).

You can verify that all the bug fixes received two code reviews.

NSPR changes: see comment 15.

NSS changes:
Bug 353475 (VC 2005 build)
Bug 352041 (coverity)
Bug 356309
Bug 357197
Bug 350200
Bug 353422 (coverity, klockwork)
Bug 115951 (coverity)
Bug 355297 (FIPS)
Bug 353896
Bug 335454
Bug 353608
Bug 354313 (memory leak)
Bug 353572 (memory leak)
Bug 353749 (FIPS)
Bug 356073 (FIPS)
Bug 354900 (FIPS)
Bug 353910 (memory leak)
Bug 127960
Bug 359484

Comment 17

[Daniel Veditz [:dveditz]]

Comment on attachment 246042 [details] [diff] [review]
Patch for the MOZILLA_1_8_BRANCH to use NSPR 4.6.4 and NSS 3.11.4

r=dveditz
approved for 1.8 branch, a=dveditz for drivers

Comment 18

[Wan-Teh Chang]

Attachment: Patch for the Mozilla trunk to use NSS 3.11.4

To satisfy the trunk testing requirement, I checked in this patch
on the Mozilla trunk first.

Checking in client.mk;
/cvsroot/mozilla/client.mk,v  <--  client.mk
new revision: 1.302; previous revision: 1.301
done

The Mozilla trunk is using NSPRPUB_PRE_4_2_CLIENT_BRANCH, which is
NSPR 4.7 pre-release and newer than NSPR 4.6.4.  Therefore I didn't
change NSPR_CO_TAG in this patch.

Comment 19

[Wan-Teh Chang]

I checked in the patch for the MOZILLA_1_8_BRANCH.

Checking in client.mk;
/cvsroot/mozilla/client.mk,v  <--  client.mk
new revision: 1.245.2.26; previous revision: 1.245.2.25
done

Comment 20

[Wan-Teh Chang]

The patch for the MOZILLA_1_8_BRANCH broke the tinderboxes because
the tinderboxes check out the NSPR CVS tag with the -D flag.  I
actually thought of this potential problem and filed bug 361332,
but I didn't realize I would encounter this bug in action.

Comment 21

[Wan-Teh Chang]

Since I wasn't sure if I will receive the necessary code reviews
for my patch in bug 361332 this week, at 17:55 PST (-0800) yesterday
(2006-11-20) I landed NSPR 4.6.4 and NSS 3.11.4 on the MOZILLA_1_8_BRANCH.
This interim measure ensures that Firefox 2.0.0.1 will use an official
NSS release.

Comment 22

[Reed Loden [:reed]]

(In reply to comment #21)
> Since I wasn't sure if I will receive the necessary code reviews
> for my patch in bug 361332 this week, at 17:55 PST (-0800) yesterday
> (2006-11-20) I landed NSPR 4.6.4 and NSS 3.11.4 on the MOZILLA_1_8_BRANCH.
> This interim measure ensures that Firefox 2.0.0.1 will use an official
> NSS release.

So, does this mean this bug is now completely fixed for the 1.8 branch?

Comment 23

[Wan-Teh Chang]

No, this bug isn't completely fixed until the patches in this bug
and bug 361332 have been checked in.  I am waiting for the necessary
code reviews for the patch for bug 361332.

Comment 24

[Wan-Teh Chang]

I checked in the patch for the MOZILLA_1_8_BRANCH.

Checking in client.mk;
/cvsroot/mozilla/client.mk,v  <--  client.mk
new revision: 1.245.2.28; previous revision: 1.245.2.27
done

Comment 25

[Doug Shelton]

(In reply to comment #24)
> I checked in the patch for the MOZILLA_1_8_BRANCH.
> 
> Checking in client.mk;
> /cvsroot/mozilla/client.mk,v  <--  client.mk
> new revision: 1.245.2.28; previous revision: 1.245.2.27
> done
> 
This appears to have broken branch builds for BeOS, with the following error:
make[7]: Entering directory `/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/nsprpub/pr/src/io'
gcc -o prfdcach.o -c      -pipe -O3 -march=i686 -mcpu=i686   -UDEBUG  -DMOZILLA_CLIENT=1 -DNDEBUG=1 -DXP_BEOS=1 -DBeOS=1 -DBEOS=1 -D_POSIX_SOURCE=1 -DHAVE_STRERROR=1  -DFORCE_PR_LOG -D_PR_PTHREADS -UHAVE_CVAR_BUILT_ON_SEM -D_NSPR_BUILD_ -I/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/dist/include/nspr -I/boot/home/develop/FirefoxTwoOh/mozilla/nsprpub/pr/include -I/boot/home/develop/FirefoxTwoOh/mozilla/nsprpub/pr/include/private  /boot/home/develop/FirefoxTwoOh/mozilla/nsprpub/pr/src/io/prfdcach.c
In file included from /boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/dist/include/nspr/md/prosdep.h:164,
                 from /boot/home/develop/FirefoxTwoOh/mozilla/nsprpub/pr/include/private/primpl.h:78,
                 from /boot/home/develop/FirefoxTwoOh/mozilla/nsprpub/pr/src/io/prfdcach.c:38:
/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/dist/include/nspr/md/_pth.h:154: #error "pthreads is not supported for this architecture"
/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/dist/include/nspr/md/_pth.h:267: #error "pthreads is not supported for this architecture"
/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/dist/include/nspr/md/_pth.h:300: #error "Need to define _PT_PTHREAD_YIELD for this platform"
make[7]: *** [prfdcach.o] Error 1
make[7]: Leaving directory `/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/nsprpub/pr/src/io'
make[6]: *** [export] Error 2
make[6]: Leaving directory `/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/nsprpub/pr/src'
make[5]: *** [export] Error 2
make[5]: Leaving directory `/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/nsprpub/pr'
make[4]: *** [export] Error 2
make[4]: Leaving directory `/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta/nsprpub'
make[3]: *** [nspr] Error 2
make[3]: Leaving directory `/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta'
make[2]: *** [default] Error 2
make[2]: Leaving directory `/boot/home/develop/FirefoxTwoOh/mozilla/opt-FF2Zeta'
make[1]: *** [build] Error 2
make[1]: Leaving directory `/boot/home/develop/FirefoxTwoOh/mozilla'
make: *** [build] Error 2
$  

Either my build environment is broken or this NSPR doesn't recognize BeOS and tries instead to build using pthreads instead of BeOS' bthreads code.

Comment 26

[Wan-Teh Chang]

Doug: none of the changes in mozilla/nsprpub between
FIREFOX_2_0_RELEASE and NSPR_4_6_4_RTM affects BeOS.

Comment 27

[Doug Shelton]

(In reply to comment #26)
> Doug: none of the changes in mozilla/nsprpub between
> FIREFOX_2_0_RELEASE and NSPR_4_6_4_RTM affects BeOS.
> 

Please excuse my stupidity on this one.  A software update unrelated to Mozilla slipped an experimental (and apparently not very functional) pthreads implementation onto my system.  This confused the build system and caused the error.  The bezilla team will need to consider whether to introduce checks to make sure we use bthreads unless we explicitly state otherwise.

Comment 28

[Wan-Teh Chang]

Doug: please open an NSPR bug on the BeOS pthreads issue.
The NSPR macro _PR_PTHREADS can only be used on Unix.  So
the code related to the _HAVE_PTHREADS test in
mozilla/nspprub/configure.in needs to be disabled for BeOS.

Comment 29

[Wan-Teh Chang]

Attachment: Patch for the MOZILLA_1_8_0_BRANCH to use NSPR 4.6.4 and NSS 3.11.4

This patch also includes the patches for the pre-requisites:
- bug 317620: to add the new freebl3.dll, freebl3.chk files
  of NSS 3.11.x to the various Mozilla installer and package
  files.
- bug 361332: allow mozilla/client.mk to pull static NSPR and
  NSS tags without the -D <date> flags.

Note that I excluded the patch for bug 288647, which was checked
in at the same time as the patch for bug 317620 but is really
independent.

This patch is just the MOZILLA_1_8_BRANCH patches backported to
the MOZILLA_1_8_0_BRANCH.  The original MOZILLA_1_8_BRANCH all have
the necessary code reviews, so I hope we don't need to review this
patch again.  (I have reviewed it thoroughly.)  I only had to tweak
the changes to two files for the MOZILLA_1_8_0_BRANCH:
- mozilla/security/manager/Makefile.in
- mozilla/toolkit/mozapps/installer/packager.mk

Comment 30

[Daniel Veditz [:dveditz]]

The danger in 1.8.0.10 is not the NSS changes themselves but making sure all the right bits make in into the packaging/release steps. The install changes are part of the patch, is there any hidden part of the release or update process that needs to know about the new file as well?

Comment 31

[Wan-Teh Chang]

Comment on attachment 248835 [details] [diff] [review]
Patch for the MOZILLA_1_8_0_BRANCH to use NSPR 4.6.4 and NSS 3.11.4

I'll be happy to work with the Mozilla release engineers to
test this patch.  I did my best to examine the MOZILLA_1_8_BRANCH
and MOZILLA_1_8_0_BRANCH trees checked out with MOZ_CO_PROJECT=all.
There are two issues not yet addressed by my patch

1. The patch for Camino bug 328522 needs to be backported
to the MOZILLA_1_8_0_BRANCH.  I have requested help in that
bug.

2. If Minimo developers are still using the MOZILLA_1_8_0_BRANCH,
two files may need to be updated to add the new freebl3.dll file:
- mozilla/minimo/config/wince_package.sh
- mozilla/minimo/base/Minimo.cpp

Bug 362404 is the only known regression of NSS 3.11.4.
NSS 3.11.4 fails to initialize on Windows 95 OSR2.x
computers that were installed by manually removing IE
from the installer.  I marked that bug WONTFIX because of
the small number of such computers in use today but prepared
a response statement with suggested workarounds.  If Firefox
1.5.0.x officially supports Windows 95, you should review
my response statement in bug 362404 and bug 362404 comment 31.

Comment 32

[Wan-Teh Chang]

Doug Turner says Minimo does not build nor is supported on
MOZILLA_1_8_0_BRANCH.  So we don't need to update those two
files in mozilla/minimo.

Comment 33

[Daniel Veditz [:dveditz]]

Comment on attachment 248835 [details] [diff] [review]
Patch for the MOZILLA_1_8_0_BRANCH to use NSPR 4.6.4 and NSS 3.11.4

approved for 1.8.0 branch, a=dveditz for drivers

Comment 34

[Wan-Teh Chang]

Attachment: Patch for the MOZILLA_1_8_0_BRANCH to use NSPR 4.6.4 and NSS 3.11.4 (as checked in)

I checked in this patch on the MOZILLA_1_8_0_BRANCH for 1.8.0.10.

This patch includes the Camino Xcode project file change (bug 328522).
I also modified mozilla/minimo/config/wince_package.sh.

Comment 35

[Jay Patel [:jay]]

v.fixed for 1.8.0 branch based on latest comments.