Last Comment Bug 858231 - Update Mozilla 23 to use NSS 3.15 (uses a new directory layout)
: Update Mozilla 23 to use NSS 3.15 (uses a new directory layout)
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: Security: PSM (show other bugs)
: Trunk
: All All
: -- normal (vote)
: mozilla24
Assigned To: Brian Smith (:briansmith, :bsmith, use NEEDINFO?)
:
Mentors:
Depends on:
Blocks: 700693
  Show dependency treegraph
 
Reported: 2013-04-04 13:44 PDT by Kai Engert (:kaie)
Modified: 2013-06-26 08:09 PDT (History)
12 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
+
fixed
fixed


Attachments
patch to adjust dbm build rules (1.78 KB, patch)
2013-04-04 13:44 PDT, Kai Engert (:kaie)
no flags Details | Diff | Splinter Review
Adapt security/build rules to new NSS layout (2.17 KB, patch)
2013-04-11 11:36 PDT, Mike Hommey [:glandium]
brian: review+
Details | Diff | Splinter Review
Refine HAVE_FREEBL_LIBS_32 into HAVE_FREEBL_LIBS_32INT32 and HAVE_FREEBL_LIBS_32FPU (1.07 KB, patch)
2013-04-19 16:08 PDT, Wan-Teh Chang
ginn.chen: review+
wtc: checkin+
Details | Diff | Splinter Review
Upgrade to NSS_3_15_BETA2 (197.12 KB, patch)
2013-04-29 16:25 PDT, Wan-Teh Chang
kaie: review+
wtc: checkin+
Details | Diff | Splinter Review
final for mozilla-central (9.76 KB, patch)
2013-06-05 07:52 PDT, Kai Engert (:kaie)
wtc: review+
Details | Diff | Splinter Review
final for mozilla-aurora 23 (83.60 KB, patch)
2013-06-05 07:53 PDT, Kai Engert (:kaie)
wtc: review+
lukasblakk+bugs: approval‑mozilla‑aurora+
Details | Diff | Splinter Review

Description Kai Engert (:kaie) 2013-04-04 13:44:47 PDT
Created attachment 733530 [details] [diff] [review]
patch to adjust dbm build rules

The NSS team is working on NSS version 3.15, which will use a new layout for its directories. The client.py script has already been updated to pull from the new location in bug 853775.

As a first step for this task, mozilla-central should pull the NSS_3_15_BETA1 tag.

I already did some local experimentation, and I can propose steps to perform the landing. Also, it seems (luckily) the beta1 tag makes the currently local patches in mozilla-central obsolete.

I propose:
- rm -rf dbm security/dbm security/coreconf
- rm security/patches/*.patch
- hg addremove

HG discovers that some of the deletions are moves.

But even for the files where hg doesn't detect the move, I don't think it matters. History for NSS is contained in the upstream NSS repository anyway.

On top of above, I used a small patch to security/build that allowed my local Linux build to pass the NSS directory. I'll attach that patch. I don't know if it's sufficient. You might want to test on the other platforms prior to landing this patch.
Comment 1 Mike Hommey [:glandium] 2013-04-04 14:02:48 PDT
Comment on attachment 733530 [details] [diff] [review]
patch to adjust dbm build rules

Review of attachment 733530 [details] [diff] [review]:
-----------------------------------------------------------------

::: security/build/Makefile.in
@@ +298,5 @@
>  endif
>  
>  NSS_DIRS =
>  ifndef NSS_DISABLE_DBM
> +NSS_DIRS += nss/lib/dbm

I think you can remove this, although you probably need to do something for DBM not to be built with NSS_DISABLE_DBM is set (which it is on android)
Comment 2 Wan-Teh Chang 2013-04-04 15:45:18 PDT
Comment on attachment 733530 [details] [diff] [review]
patch to adjust dbm build rules

Review of attachment 733530 [details] [diff] [review]:
-----------------------------------------------------------------

::: security/build/Makefile.in
@@ +298,5 @@
>  endif
>  
>  NSS_DIRS =
>  ifndef NSS_DISABLE_DBM
> +NSS_DIRS += nss/lib/dbm

Yes, we should simply remove this (lines 301-303).

NSS_DISABLE_DBM is now handled by nss/lib/{Makefile,manifest.mn}.

@@ -440,5 @@
>  libs-nss/lib/softoken: $(NSPR_IMPORT_LIBS) $(SQLITE_IMPORT_LIB)
>  libs-nss/lib/softoken: libs-nss/lib/freebl
> -ifndef NSS_DISABLE_DBM
> -libs-nss/lib/softoken: libs-dbm
> -endif

Did you delete this because libs-dbm isn't defined?

http://mxr.mozilla.org/mozilla-central/search?string=libs-dbm
Comment 3 Mike Hommey [:glandium] 2013-04-11 11:36:07 PDT
Created attachment 736404 [details] [diff] [review]
Adapt security/build rules to new NSS layout

This should work.
Comment 4 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2013-04-11 17:08:48 PDT
https://hg.mozilla.org/integration/mozilla-inbound/rev/0857f2bc8f8a
https://hg.mozilla.org/integration/mozilla-inbound/rev/c4bac10ee49e

In addition to following Kai's instructions and merging glandium's changes, I updated configure.in to require NSS 3.15 and I updated security/patches/README to remove the references to the no-longer-needed private patches that we previously had in mozilla-central.

Thanks for the help with this!
Comment 6 Kai Engert (:kaie) 2013-04-12 05:13:25 PDT
Reopening, because that was only a beta, but this bug is track the full landing including the final release of NSS 3.15 (pending).
Comment 7 Wan-Teh Chang 2013-04-16 15:32:07 PDT
Comment on attachment 736404 [details] [diff] [review]
Adapt security/build rules to new NSS layout

Review of attachment 736404 [details] [diff] [review]:
-----------------------------------------------------------------

::: security/build/Makefile.in
@@ +302,5 @@
>  NSS_DIRS += nss/lib
> +else
> +ifndef NSS_DISABLE_DBM
> +NSS_DIRS += nss/lib/dbm
> +endif

I think it is better to add nss/lib/dbm here:

336 # Remaining nss/lib directories
337 NSS_DIRS += nss/lib/freebl nss/lib/softoken nss/lib/jar nss/lib/crmf nss/lib/ckfw nss/lib/libpkix
Comment 8 Ginn Chen 2013-04-18 05:09:07 PDT
In security/nss/lib/freebl/Makefile, HAVE_ABI32_INT32, USE_ABI32_INT32 sections under SPARC 32bit are removed.

Is it intentional?

But in security/build/Makefile.in, it still have
HAVE_FREEBL_LIBS_32 = 1

It caused build failure.
Comment 9 Wan-Teh Chang 2013-04-19 15:47:40 PDT
Ginn: yes, we dropped Solaris SPARC v8 support in NSS 3.15 (bug 841664).

Could you create a patch for security/build/Makefile.in? You may just
need to delete the line
  HAVE_FREEBL_LIBS_32 = 1

I have another question for you: which versions of the Solaris Studio
compiler do you use? There are a lot of compiler warnings when NSS is
compiled with the latest version of the Solaris Studio compiler. It
will be easier to fix those warnings if I only need to support recent
versions of Solaris Studio.
Comment 10 Wan-Teh Chang 2013-04-19 15:56:56 PDT
Ginn: I looked into the security/build/Makefile.in problem. I think
HAVE_FREEBL_LIBS_32 needs to be refined into two variables:

  HAVE_FREEBL_LIBS_32INT32
  HAVE_FREEBL_LIBS_32FPU

and

  ifdef HAVE_FREEBL_LIBS_32INT32
  NSS_EXTRA_DLLS += freebl_32int_3
  endif
  ifdef HAVE_FREEBL_LIBS_32FPU
  NSS_EXTRA_DLLS += freebl_32fpu_3
  endif
Comment 11 Wan-Teh Chang 2013-04-19 16:08:43 PDT
Created attachment 739871 [details] [diff] [review]
Refine HAVE_FREEBL_LIBS_32 into HAVE_FREEBL_LIBS_32INT32 and HAVE_FREEBL_LIBS_32FPU

Ginn: please review and test this patch. If it works, please check it in
for me. Thanks.

Description: Refine HAVE_FREEBL_LIBS_32 into HAVE_FREEBL_LIBS_32INT32 and
HAVE_FREEBL_LIBS_32FPU, and set only HAVE_FREEBL_LIBS_32FPU to 1 for
32-bit Solaris SPARC builds.
Comment 12 Ginn Chen 2013-04-21 01:52:27 PDT
(In reply to Wan-Teh Chang from comment #9)

> I have another question for you: which versions of the Solaris Studio
> compiler do you use? There are a lot of compiler warnings when NSS is
> compiled with the latest version of the Solaris Studio compiler. It
> will be easier to fix those warnings if I only need to support recent
> versions of Solaris Studio.

I'm using Solaris Studio 12.2 and 12.3 for Firefox.
I guess another team is compiling NSS with Studio 12.1.

IMO, you only need to consider 12.3.
If it breaks 12.1 or 12.2, they should apply workarounds on their local workspace.
Comment 14 Kai Engert (:kaie) 2013-04-22 05:24:26 PDT
I've tagged NSS_3_15_BETA2.
Could you please pick it up in mozilla-central?
Comment 15 Ryan VanderMeulen [:RyanVM] 2013-04-22 05:35:17 PDT
https://hg.mozilla.org/mozilla-central/rev/a1a2af3cebff
Comment 16 Wan-Teh Chang 2013-04-29 16:25:04 PDT
Created attachment 743333 [details] [diff] [review]
Upgrade to NSS_3_15_BETA2

Kai, this is the patch to push NSS_3_15_BETA2 to m-c.

I noticed that Brian did not add the .cvsignore files when he pushed
NSS_3_15_BETA1, so I didn't add those files either.

https://hg.mozilla.org/integration/mozilla-inbound/rev/08e868f22c98
Comment 17 Wan-Teh Chang 2013-04-29 16:30:00 PDT
Comment on attachment 743333 [details] [diff] [review]
Upgrade to NSS_3_15_BETA2

Kai, I saw these in the diffs:

>diff --git a/security/nss/cmd/pk11gcmtest/Makefile b/security/nss/cmd/pk11gcmtest/Makefile
>old mode 100644
>new mode 100755

>diff --git a/security/nss/tests/libpkix/certs/make-ca-u50-u51 b/security/nss/tests/libpkix/certs/make-ca-u50-u51
>old mode 100644
>new mode 100755

Do you know why? I wonder if they are caused by files being
pushed on Windows vs. on Linux.
Comment 18 Kai Engert (:kaie) 2013-04-29 16:33:08 PDT
Comment on attachment 743333 [details] [diff] [review]
Upgrade to NSS_3_15_BETA2

I didn't review this long patch, I think it's not necessary. I assume you simply used the "python client.py update_nss" script.
Comment 19 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2013-04-29 20:39:58 PDT
(In reply to Wan-Teh Chang from comment #17)
> Comment on attachment 743333 [details] [diff] [review]
> Upgrade to NSS_3_15_BETA2
> 
> Kai, I saw these in the diffs:
> 
> >diff --git a/security/nss/cmd/pk11gcmtest/Makefile b/security/nss/cmd/pk11gcmtest/Makefile
> >old mode 100644
> >new mode 100755
> 
> >diff --git a/security/nss/tests/libpkix/certs/make-ca-u50-u51 b/security/nss/tests/libpkix/certs/make-ca-u50-u51
> >old mode 100644
> >new mode 100755
> 
> Do you know why? I wonder if they are caused by files being
> pushed on Windows vs. on Linux.

I did a little searching and I found that this does indeed seem to be caused by me doing the update on Windows. khuey (who just updated NSPR in mozilla-beta) and I both use Windows as our primary dev. platform. Until we can find a real solution, I will do the updates of NSS and NSPR in my Linux VM.

Probably we should just change all the test scripts so that the executable bit does not need to be set (i.e. explicitly invoke the interpreter when running any script). Anybody disagree? If not, I will file the bug in NSS.
Comment 20 Ed Morley [:emorley] 2013-04-30 05:58:02 PDT
https://hg.mozilla.org/mozilla-central/rev/08e868f22c98
Comment 21 Wan-Teh Chang 2013-05-01 12:38:14 PDT
NSS_3_15_BETA3 (tagged this morning) pushed to mozilla-inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/0314d200873a
Comment 22 Wan-Teh Chang 2013-05-01 12:58:28 PDT
(In reply to Brian Smith (:bsmith) from comment #19)
>
> Probably we should just change all the test scripts so that the executable
> bit does not need to be set (i.e. explicitly invoke the interpreter when
> running any script). Anybody disagree? If not, I will file the bug in NSS.

This seems like an over-reaction. Do you know why nss/tests/all.sh, which is
marked as executable, doesn't have the permission mode problem when you pushed
it to mozilla-central on Windows?
Comment 23 Ryan VanderMeulen [:RyanVM] 2013-05-01 13:06:26 PDT
(In reply to Wan-Teh Chang from comment #21)
> NSS_3_15_BETA3 (tagged this morning) pushed to mozilla-inbound:
> https://hg.mozilla.org/integration/mozilla-inbound/rev/0314d200873a

Backed out for Windows build bustage.
https://hg.mozilla.org/integration/mozilla-inbound/rev/05275490b9c1

https://tbpl.mozilla.org/php/getParsedLog.php?id=22468935&tree=Mozilla-Inbound
Comment 24 Wan-Teh Chang 2013-05-02 16:21:16 PDT
NSS_3_15_BETA3 with two local patches (bug-835919.patch and
bug-832272.patch) pushed to mozilla-inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c6f5c1bbcf76
Comment 25 Ed Morley [:emorley] 2013-05-03 08:45:44 PDT
https://hg.mozilla.org/mozilla-central/rev/c6f5c1bbcf76
Comment 26 Wan-Teh Chang 2013-05-10 17:21:30 PDT
NSS_3_15_BETA4 pushed to mozilla-inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/06e3d7e19a09
Comment 27 David Baron :dbaron: ⌚️UTC+2 (review requests must explain patch) 2013-05-11 22:49:23 PDT
https://hg.mozilla.org/mozilla-central/rev/06e3d7e19a09
Comment 28 Wan-Teh Chang 2013-05-20 15:28:48 PDT
NSS_3_15_BETA5 pushed to mozilla-inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/cc8bbdd0533c
Comment 29 Wan-Teh Chang 2013-05-20 16:34:07 PDT
I had to revert to NSS_3_15_BETA4 in mozilla-inbound
because NSS_3_15_BETA5 broke three Mozilla build bots:
https://hg.mozilla.org/integration/mozilla-inbound/rev/cd4d5caefa88

I filed bug 874261 for the Mozilla build system problem.
Comment 30 Wan-Teh Chang 2013-05-20 19:17:53 PDT
Second attempt at pushing NSS_3_15_BETA5 to mozilla-inbound, without
Kai's SECITEM_ReallocItemV2 patch this time to work around bug 874261:
https://hg.mozilla.org/integration/mozilla-inbound/rev/011a86ed18fe
Comment 31 Ryan VanderMeulen [:RyanVM] 2013-05-21 10:36:22 PDT
https://hg.mozilla.org/mozilla-central/rev/011a86ed18fe
Comment 32 Wan-Teh Chang 2013-05-22 16:37:55 PDT
NSS_3_15_BETA6 pushed to mozilla-inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/84985c77b141

To work around bug 874261, I applied a private patch to avoid using
the new SECITEM_ReallocItemV2 function.
Comment 33 Ed Morley [:emorley] 2013-05-23 05:01:05 PDT
https://hg.mozilla.org/mozilla-central/rev/84985c77b141
Comment 34 Kai Engert (:kaie) 2013-05-23 06:08:46 PDT
undoing:
  status-firefox24: --- → fixed
because the main change hasn't landed yet
Comment 35 Kai Engert (:kaie) 2013-06-05 07:50:00 PDT
Final versions of NSS 3.15 and NSPR 4.10 have been released.

We must upgrade mozilla-central (24) and mozilla-aurora (23) to pick them up.
Comment 36 Kai Engert (:kaie) 2013-06-05 07:52:29 PDT
Created attachment 758583 [details] [diff] [review]
final for mozilla-central

mozilla-central currently uses
NSPR (portable runtime) version: NSPR_4_10_BETA2
NSS (security library) version: NSS_3_15_BETA6

This patch was produced using:

python client.py update_nspr NSPR_4_10_RTM
python client.py update_nss NSS_3_15_RTM
Comment 37 Kai Engert (:kaie) 2013-06-05 07:53:52 PDT
Created attachment 758585 [details] [diff] [review]
final for mozilla-aurora 23
Comment 38 Kai Engert (:kaie) 2013-06-05 07:54:48 PDT
Comment on attachment 758585 [details] [diff] [review]
final for mozilla-aurora 23

approval required because of the rule
  "must ship final versions of nspr/nss in releases"
Comment 39 Wan-Teh Chang 2013-06-05 11:55:58 PDT
Comment on attachment 758583 [details] [diff] [review]
final for mozilla-central

Review of attachment 758583 [details] [diff] [review]:
-----------------------------------------------------------------

r=wtc.

IMPORTANT: please remove the local patch security/patches/revert-bug-808217.patch
and remove the description of that patch in security/patches/README.

Watch out for linker errors (unresolved symbol SECITEM_ReallocItemV2) on the
Android and B2G build bots.
Comment 40 Wan-Teh Chang 2013-06-05 12:05:02 PDT
Comment on attachment 758585 [details] [diff] [review]
final for mozilla-aurora 23

Review of attachment 758585 [details] [diff] [review]:
-----------------------------------------------------------------

r=wtc.

IMPORTANT: change nsprpub/config/prdepend.h.
Comment 41 Kai Engert (:kaie) 2013-06-05 12:10:11 PDT
(In reply to Wan-Teh Chang from comment #39)
> IMPORTANT: please remove the local patch
> security/patches/revert-bug-808217.patch
> and remove the description of that patch in security/patches/README.

https://hg.mozilla.org/integration/mozilla-inbound/rev/6486354e07e4

I read your comment, but then forgot to do it at the same time of the landing.
And immediately after I checked in, mozilla-inbound was closed because of other bustage :)

This cleanup is for tracking purposes, only, and doesn't affect the build.
I'll do it after the tree opens.
Comment 42 Kai Engert (:kaie) 2013-06-05 12:11:33 PDT
(In reply to Wan-Teh Chang from comment #40)
> 
> IMPORTANT: change nsprpub/config/prdepend.h.

I wonder why the client.py script didn't make that change on the aurora branch...
Comment 43 Kai Engert (:kaie) 2013-06-05 17:09:04 PDT
> Watch out for linker errors (unresolved symbol SECITEM_ReallocItemV2) on the
> Android and B2G build bots.

We didn't get such errors after the landing. We are fine.

> IMPORTANT: please remove the local patch
> security/patches/revert-bug-808217.patch
> and remove the description of that patch in security/patches/README.

done:
https://hg.mozilla.org/integration/mozilla-inbound/rev/796a79fc347a


The work for mozilla-central is done.

It's OK to set this to fixed, after inbound has been merged into central.
Comment 45 Ryan VanderMeulen [:RyanVM] 2013-06-07 13:08:01 PDT
https://hg.mozilla.org/releases/mozilla-aurora/rev/5cd24379d192

I forgot to set the author before pushing :(. Please make sure your attached patches have it in the future.
Comment 46 Oleg Romashin (:romaxa) 2013-06-26 08:09:06 PDT
http://hg.mozilla.org/mozilla-central/rev/c4bac10ee49e#l1.12
Native NSS version check does not work 
AM_PATH_NSS - need version specified in format \D+.\D+.\D+
otherwise it would not detect and think 3.14.3 - as 3.15 - compatible

Note You need to log in before you can comment on or make changes to this bug.