Last Comment Bug 323977 - Crash when build runs shlibsign on FreeBSD 5.4
: Crash when build runs shlibsign on FreeBSD 5.4
Status: VERIFIED FIXED
:
Product: NSS
Classification: Components
Component: Libraries (show other bugs)
: 3.11
: x86 FreeBSD
: P2 normal (vote)
: 3.11.1
Assigned To: Wan-Teh Chang
: Jason Reid
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2006-01-18 21:45 PST by Glenn Randers-Pehrson
Modified: 2006-01-21 17:44 PST (History)
5 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Proposed patch (777 bytes, patch)
2006-01-19 04:57 PST, Nelson Bolyard (seldom reads bugmail)
no flags Details | Diff | Review
Patch to reproduce this crash on Linux (835 bytes, patch)
2006-01-19 10:22 PST, Wan-Teh Chang
no flags Details | Diff | Review
Patch 1: link libfreebl3.so with -Bsymbolic on FreeBSD (582 bytes, patch)
2006-01-19 10:24 PST, Wan-Teh Chang
nelson: review+
Details | Diff | Review
Patch 2: use mapfile files (*.def) on FreeBSD (843 bytes, patch)
2006-01-19 10:30 PST, Wan-Teh Chang
nelson: review+
glennrp+bmo: review+
Details | Diff | Review
Patch 1: link libfreebl3.so with -Bsymbolic, v2 (1.00 KB, patch)
2006-01-19 13:52 PST, Wan-Teh Chang
nelson: review+
Details | Diff | Review

Description Glenn Randers-Pehrson 2006-01-18 21:45:23 PST
This is a spinoff from bug #317620.

While building seamonkey from trunk on a FreeBSD-5.4 platform, the build dumps core while attempting to run shlibsign.

$ uname -a
FreeBSD [host].org 5.4-RELEASE FreeBSD 5.4-RELEASE #2: Mon Jun 13 08:42:36
   CDT 2005 [user]@[host].org:/usr/obj/usr/src/sys/DELLSMP54  i386

$ gcc -v
Using built-in specs.
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 3.4.2 [FreeBSD] 200407

$ ld -v
GNU ld version 2.15 [FreeBSD] 2004-05-23

$ ld -V
GNU ld version 2.15 [FreeBSD] 2004-05-23
  Supported emulations:
   elf_i386_fbsd

Stack trace from mozilla/security/nss/cmd/shlibsign/shlibsign.core:

#0  0x280a833f in RNG_RNGInit ()
   from /scratch/glennrp/Mozilla/Mozilla-1.9a1-060117/mozilla/dist/lib/libssl3.s
o
#1  0x280a836e in RNG_RNGInit ()
   from /scratch/glennrp/Mozilla/Mozilla-1.9a1-060117/mozilla/dist/lib/libssl3.s
o
#2  0x280a836e in RNG_RNGInit ()
   from /scratch/glennrp/Mozilla/Mozilla-1.9a1-060117/mozilla/dist/lib/libssl3.s
o
#3  0x280a836e in RNG_RNGInit ()
   from /scratch/glennrp/Mozilla/Mozilla-1.9a1-060117/mozilla/dist/lib/libssl3.s
o

ad infinitum.
Comment 1 Nelson Bolyard (seldom reads bugmail) 2006-01-19 03:06:07 PST
This means that libfreebl*.so was/were not built with -Bsymbolic.
Since I don't have a FreeBSD system on which to build, let me tell you
the general idea, and ask you to implement it and report the results.

Look at http://lxr.mozilla.org/security/source/security/nss/lib/freebl/Makefile#191

There you will see how this is done on Solaris.  In particular, for GCC,
when using gnu ld, we add some command line args.  e.g.
   MKSHLIB += -Wl,-Bsymbolic,-z,now,-z,text

I suggest adding a section in that same Makefile, at about line 205 (right
after the SunOS section) that looks something like this:

ifeq ($(OS_TARGET),FreeBSD)
   MKSHLIB += -Wl,-Bsymbolic,-z,now,-z,text
endif

and then gmake clean in lib/freebl, and then remake nss.  
This should fix it.  
It MAY cause new unresolved symbols errors in linking libfreebl*.so.
If that happens, show us all the linker errors in linking that DSO.
Comment 2 Glenn Randers-Pehrson 2006-01-19 04:42:17 PST
Nelson: with your change the build doesn't crash.  Instead, shlibsign generates
some key pairs and then the build proceeds.  I didn't see any complaints about
unsatisfied external references.
Comment 3 Nelson Bolyard (seldom reads bugmail) 2006-01-19 04:57:28 PST
Created attachment 208959 [details] [diff] [review]
Proposed patch

Is this the change you made?
Comment 4 Glenn Randers-Pehrson 2006-01-19 05:19:45 PST
I added the lines at Makefile line 359, outside the SunOS block.
Comment 5 Glenn Randers-Pehrson 2006-01-19 07:42:21 PST
The "-z,text" is ignored by FreeBSD, according to "man ld" and can be omitted.
Comment 6 Wan-Teh Chang 2006-01-19 10:22:21 PST
Created attachment 208982 [details] [diff] [review]
Patch to reproduce this crash on Linux

I can reproduce this crash on Linux (2.6 kernel) with
this patch.  This patch makes us build shared libraries
on Linux the same way we build shared libraries on
FreeBSD: without using a GNU ld version script to control
exported symbols.  With a debug build of the NSS_3_11_BRANCH,
I get this stack trace:

(gdb) where 10
#0  0x0037b90a in RNG_RNGInit () at loader.c:943
#1  0x0037b93f in RNG_RNGInit () at loader.c:946
#2  0x0037b93f in RNG_RNGInit () at loader.c:946
#3  0x0037b93f in RNG_RNGInit () at loader.c:946
#4  0x0037b93f in RNG_RNGInit () at loader.c:946
#5  0x0037b93f in RNG_RNGInit () at loader.c:946
#6  0x0037b93f in RNG_RNGInit () at loader.c:946
#7  0x0037b93f in RNG_RNGInit () at loader.c:946
#8  0x0037b93f in RNG_RNGInit () at loader.c:946
#9  0x0037b93f in RNG_RNGInit () at loader.c:946
(More stack frames follow...)
(gdb) where -20
#775069 0x0037b93f in RNG_RNGInit () at loader.c:946
#775070 0x0037b93f in RNG_RNGInit () at loader.c:946
#775071 0x0037b93f in RNG_RNGInit () at loader.c:946
#775072 0x0037b93f in RNG_RNGInit () at loader.c:946
#775073 0x0037b93f in RNG_RNGInit () at loader.c:946
#775074 0x0037b93f in RNG_RNGInit () at loader.c:946
#775075 0x0037b93f in RNG_RNGInit () at loader.c:946
#775076 0x0037b93f in RNG_RNGInit () at loader.c:946
#775077 0x0037b93f in RNG_RNGInit () at loader.c:946
#775078 0x0037b93f in RNG_RNGInit () at loader.c:946
#775079 0x0037b93f in RNG_RNGInit () at loader.c:946
#775080 0x00a257e3 in nsc_CommonInitialize (pReserved=0xbffd3b40, isFIPS=0)
    at pkcs11.c:2972
#775081 0x00a259d9 in NSC_Initialize (pReserved=0xbffd3b40) at pkcs11.c:3052
#775082 0x00873ff9 in secmod_ModuleInit (mod=0x8660538,
    alreadyLoaded=0xbffd3ba4) at pk11load.c:150
#775083 0x00874486 in SECMOD_LoadPKCS11Module (mod=0x8660538)
    at pk11load.c:322
#775084 0x0087e9b2 in SECMOD_LoadModule (
    modulespec=0x86602b0 "library= name=\"NSS Internal PKCS #11 Module\" paramet
ers=\"configdir='' certPrefix='' keyPrefix='' secmod='' flags=readOnly,noCertDB,
noModDB,forceOpen,optimizeSpace \" NSS=\"Flags=internal,critical trustO"..., par
ent=0x8660000, recurse=1) at pk11pars.c:323
#775085 0x0087ea26 in SECMOD_LoadModule (
    modulespec=0x865f578 "name=\"NSS Internal Module\" parameters=\"configdir=''
 certPrefix='' keyPrefix='' secmod='' flags=readOnly,noCertDB,noModDB,forceOpen,
optimizeSpace \" NSS=\"flags=internal,moduleDB,moduleDBOnly,critical\"",
    parent=0x0, recurse=1) at pk11pars.c:338
#775086 0x00850706 in nss_Init (configdir=0x8c474e "",
    certPrefix=0x8c474e "", keyPrefix=0x8c474e "", secmodName=0x8c474e "",
    readOnly=1, noCertDB=1, noModDB=1, forceOpen=1, noRootInit=1,
    optimizeSpace=1, noSingleThreadedModules=0,
    allowAlreadyInitializedModules=0, dontFinalizeModules=0) at nssinit.c:467
#775087 0x00850925 in NSS_NoDB_Init (configdir=0x805428a "") at nssinit.c:584
#775088 0x0804b112 in main (argc=4, argv=0xbffd5174) at shlibsign.c:224
(gdb)

The mozilla/security/nss/lib/freebl/loader.c file is
rev. 1.26.2.1.  The relevent code is reproduced here:

   940
   941 SECStatus
   942 RNG_RNGInit(void)
   943 {
   944   if (!vector && PR_SUCCESS != freebl_RunLoaderOnce())
   945       return SECFailure;
   946   return (vector->p_RNG_RNGInit)();
   947 }

With this stack trace, Glenn's stack trace, and the
knowledge that libssl3.so, libsoftokn3.so, and libfreebl.so
all export the symbol RNG_RNGInit and shlibsign is
linked with the NSS libraries in this order:
    -lssl3 -lsmime3 -lnss3
we can conclude that it is the copy of RNG_RNGInit in
libssl3.so (the first in the linking order) that gets
used in all libraries, and this caused the infinite
recursion:
    RNG_RNGInit in libssl3.so
 -> vector->p_RNG_RNGInit (a function pointer variable
    in libfreebl3.so)
 -> RNG_RNGInit (a reference in libfreebl3.so.
                 intended target: RNG_RNGInit in libfreebl3.so.
                 actual target: RNG_RNGInit in libssl3.so)

So we need to make the RNG_RNGInit reference in libfreebl3.so
bind to the definition of RNG_RNGInit in libfreebl3.so.  I'll
attach two fixes that Glenn tested next.
Comment 7 Wan-Teh Chang 2006-01-19 10:24:53 PST
Created attachment 208983 [details] [diff] [review]
Patch 1: link libfreebl3.so with -Bsymbolic on FreeBSD

If we make this change, we should also use the -Bsymbolic
linker flag on other platforms that use the GNU ld, such
as Linux.
Comment 8 Wan-Teh Chang 2006-01-19 10:30:58 PST
Created attachment 208984 [details] [diff] [review]
Patch 2: use mapfile files (*.def) on FreeBSD

This patch kills two birds with one stone.  We should
definitely checks in this patch whether we check in
patch 1 or not.
Comment 9 Nelson Bolyard (seldom reads bugmail) 2006-01-19 11:14:36 PST
Comment on attachment 208983 [details] [diff] [review]
Patch 1: link libfreebl3.so with -Bsymbolic on FreeBSD

r=nelson
Comment 10 Wan-Teh Chang 2006-01-19 11:21:43 PST
Comment on attachment 208984 [details] [diff] [review]
Patch 2: use mapfile files (*.def) on FreeBSD

Glenn, Mikhail,

Does ld have the --version-script option on all
the FreeBSD versions we need to support?
Comment 11 Nelson Bolyard (seldom reads bugmail) 2006-01-19 11:26:00 PST
> This patch kills two birds with one stone.  

I found that the documentation for "linker scripts" on numerous platforms
does indicate that "local" symbols will be handled as they would for -Bsymbolic,
that is, a DSO's local symbols will always satisfy that DSO's own references
to those symbols.  

But in the past, we found that that was not true on at least one platform.
On Solaris, we found it necessary to do both a linker script (.def file)
and also use -Bsymbolic to avoid the problem described in this bug.

Personally, I think -Bsymbolic is desirable in a shared lib, because a DSO
generally wants to use its own symbols, whether they are global or local.

I'd suggest we do both, script and -Bsymbolic, on all relevant platforms,
but I have also read that on some platforms, -Bsymbolic requires explicitly
linking with all dependencies, direct and indirect, including (e.g.) -lc.
So, I'd say we should use scripts as widely as possible, and use -Bsymbolic
with freebl on any platform on which it doesn't cause linker problems.
Comment 12 Glenn Randers-Pehrson 2006-01-19 11:36:15 PST
  From "man ld" on FreeBSD5.4, ld 2.15:

         --version-script=version-scriptfile
           Specify the name of a version script to the linker.  This is  typi-
           cally  used  when  creating  shared libraries ...
Comment 13 Glenn Randers-Pehrson 2006-01-19 11:53:29 PST
Chang:  I tested both patches separately in a seamonkey build and in a
standalone NSS build on the FreeBSD 5.4 platform and they both worked
(i.e., the builds ran to completion instead of dumping core, and the
resulting seamonkey was useable).  I don't think I am entitled to do
official reviews though.
Comment 14 Mikhail T. 2006-01-19 11:59:02 PST
My two cents: builds of any software, that uses nspr or nss ought to rely on the devel/nspr and security/nss ports instead of building the software's own versions.

The port of seamonkey should follow the path of firefox and simply LIB_DEPEND on nss.

This not only makes the installs leaner, cleaner, and more secure, but also allows us to leverage the time spent researching and patching those two "cornerstone" packages.

Mozilla really ought to stop bundling them with each application...
Comment 15 Wan-Teh Chang 2006-01-19 13:52:27 PST
Created attachment 209007 [details] [diff] [review]
Patch 1: link libfreebl3.so with -Bsymbolic, v2

Made the patch applicable to all platforms known
to use the GNU ld.  Added a comment to explain what
why the freebl libraries need to be linked with
-Bsymbolic.
Comment 16 Glenn Randers-Pehrson 2006-01-19 19:06:33 PST
Mikhail, re comment #14: Looking at configure.in, the default behavior appears
to be already the way you are suggesting.  It only uses the embedded nss and
nspr libraries when it cannot find them on the system, or if the user explicitly
specifies "--without-system-nss" or "--without-system-nspr" to force use of the
embedded libraries.  So maybe all that's needed is some education on
how to build and use these system libraries.
Comment 17 Mikhail T. 2006-01-19 19:12:32 PST
(In reply to comment #16)
> So maybe all that's needed is some education on
> how to build and use these system libraries.

The easiest of all:

% cd /usr/ports/security/nss
% make
# make test is advised here, but is optional
% make install clean

This will also install devel/nspr, as it is listed as a dependency.
Comment 18 Glenn Randers-Pehrson 2006-01-19 20:20:49 PST
Easy if you have root permission.  Without it, naturally it doesn't
work.  I assume it's simple enough to put the nss library somewhere in
user space, though.

$ make
===>  Vulnerability check disabled, database not found
=> nss-3.9.2.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> /usr/ports/distfiles is not writable by you; cannot fetch.
*** Error code 1

Stop in /usr/ports/security/nss.
Comment 19 Glenn Randers-Pehrson 2006-01-19 23:23:54 PST
The seamonkey build creates a new libnspr4.so in my $prefix/lib/seamonkey-1.5a,
but according to "ldd seamonkey-bin" it actually uses /usr/local/lib/libnspr4.so.

It also creates libnss3.so and libsoftokn3.so but these don't show up in
the "ldd" listing at all.

It seems that my build script is doing a lot more work than it needs to.
Comment 20 Mikhail T. 2006-01-20 06:31:46 PST
(In reply to comment #18)
> Easy if you have root permission.  Without it, naturally it doesn't
> work.  I assume it's simple enough to put the nss library somewhere in
> user space, though.

Yes. You can specify an alternative location for the downloaded source codes and an alternative PREFIX:

      make DISTDIR=/home/glennrp/distfiles PREFIX=/home/glennrp build test install

(In reply to comment #19)
> The seamonkey build creates a new libnspr4.so in my
> $prefix/lib/seamonkey-1.5a,
> but according to "ldd seamonkey-bin" it actually uses
> /usr/local/lib/libnspr4.so.

Yes, this is, what I mean, when I say, it is cleaner to use each module's own port.
Comment 21 Wan-Teh Chang 2006-01-20 07:32:09 PST
Glenn, Mikhail,

We recently added --with-system-nspr and --with-system-nss
support to the Mozilla trunk.  --with-system-nspr already
existed, but it wasn't fully implemented (bug 288647).
--with-system-nss is new (bug 255408).  These two configure
options use the nspr-config and nss-config scripts to get
the locations of the NSPR and NSS headers and libraries.
I'd appreciate it if you could test the --with-system-nspr
and --with-system-nss configure options on FreeBSD.  Since
this issue is off topic for this bug report, if you discover
any problems or want to suggest changes to --with-system-nspr
and --with-system-nss for FreeBSD, please open a new bug report
against product:Core component:Build Config and cc me.
Comment 22 Nelson Bolyard (seldom reads bugmail) 2006-01-20 10:28:02 PST
Comment on attachment 209007 [details] [diff] [review]
Patch 1: link libfreebl3.so with -Bsymbolic, v2

I don't actually know that this will work on all the named platforms, but I like the comment. :)
Comment 23 Wan-Teh Chang 2006-01-20 10:34:08 PST
Comment on attachment 208984 [details] [diff] [review]
Patch 2: use mapfile files (*.def) on FreeBSD

Nelson, could you also approve this patch for the
NSS_3_11_BRANCH?  I copied the mapfile (ld version
script) related code from Linux2.6.mk.  This patch
has been tested by Glenn Randers-Pehrson (glennrp).
Comment 24 Nelson Bolyard (seldom reads bugmail) 2006-01-20 18:02:24 PST
Comment on attachment 208984 [details] [diff] [review]
Patch 2: use mapfile files (*.def) on FreeBSD

Yes.  r=nelson
Glenn, Thanks again for testing these patches!
Comment 25 Wan-Teh Chang 2006-01-20 18:41:06 PST
I checked in the patch on the NSS trunk (NSS 3.12) and
NSS_3_11_BRANCH (NSS 3.11.1), and moved the NSS_CLIENT_TAG
(Mozilla 1.9 alpha) on the two files.

Checking in coreconf/FreeBSD.mk;
/cvsroot/mozilla/security/coreconf/FreeBSD.mk,v  <--  FreeBSD.mk
new revision: 1.9; previous revision: 1.8
done
Checking in nss/lib/freebl/Makefile;
/cvsroot/mozilla/security/nss/lib/freebl/Makefile,v  <--  Makefile
new revision: 1.71; previous revision: 1.70
done

Checking in coreconf/FreeBSD.mk;
/cvsroot/mozilla/security/coreconf/FreeBSD.mk,v  <--  FreeBSD.mk
new revision: 1.8.2.1; previous revision: 1.8
done
Checking in nss/lib/freebl/Makefile;
/cvsroot/mozilla/security/nss/lib/freebl/Makefile,v  <--  Makefile
new revision: 1.70.2.1; previous revision: 1.70
done
Comment 26 Glenn Randers-Pehrson 2006-01-21 15:41:53 PST
Re comment #14, #16, #20, and #21: see bug #324283
Comment 27 Glenn Randers-Pehrson 2006-01-21 17:44:06 PST
Verified that my build on FreeBSD no longer crashes.  Thanks!

Note You need to log in before you can comment on or make changes to this bug.