77123 - Get PSM2.0 build on Solaris

Status: VERIFIED DUPLICATE of bug 76370

(Core Graveyard :: Security: UI, defect)

Description

[Margaret Chan]

Attempted to build PSM2.0 on Solaris with Forte6 using the following steps:

1. cvs co mozilla/client.mk
2. cd mozilla
3. gmake -f client.mk checkout
4. gmake -f client.mk checkout BUILD_MODULES=psm2
5. gmake -f client.mk build_all
6. add to .mozconfig the line:  ac_add_options --enable_modules=psm2
7. gmake -f client.mk build_all BUILD_MODULES=psm2

expected result:  build successfully
actual result:  build failed with errors complaining about gcc not found

Workaround:

set NS_USE_NATIVE = 1 in the ../security/coreconf/SunOS.mk file
I am not sure if setting this environment variable before the build will work
or not.  I have tried it once, but I see some other errors, so I have applied
the workaround in the file to keep the build going.

With this workaround, the build has gone further along and failed again
complaining about not able to find ../dist/SunOS5.7_sparc_DBG.OBJ.  This
directory is no where to be found.  Instead ../dist/SunOS5.7_DBG.OBJ is
there.  Directory name being used is inconsistent, thus causes the build error. 
Apply another workaround:

Replace the following line,
CORECONF_INSTALL     = $(DIST)/$(CORECONF_OBJDIR)
by 
CORECONF_INSTALL     = $(MOZ_BUILD_ROOT)/dist
in ../security/manager/Makefile.in

then rerun step 7 again.  The build finished but the browser
did not come up (Segmentation fault).  

Apply another workaround:  setenv LD_LIBRARY_PATH
.../mozilla/dist/lib:${LD_LIBRARY_PATH}
to allow bringup.  It appears that some library files
are not installed properly.

Will need proper fixes for these workarounds.

Comment 1

[Wan-Teh Chang]

Margaret: I have a fix for the NS_USE_NATIVE (gcc not found) problem.
My fix is awaiting a super review.  With my fix, you would still need
to set NS_USE_NATIVE to 1 in your environment, but at least you don't
need to manually modify any makefile.

Comment 2

[Wan-Teh Chang]

OK, I understand what caused the inconsistent use of directory name.
mozilla/security/manager/Makefile.in always sets CPU_TAG to _$(CPU_ARCH),
while NSS (mozilla/security/coreconf/ruleset.mk) uses a more complicated
rule to assign value to CPU_TAG.

One way to fix this is to just include mozilla/security/coreconf/ruleset.mk,
but I am not sure if it may conflict with the PSM2 makefile.  Another fix is
to copy just the makefile fragment that sets CPU_TAG
from mozilla/security/coreconf/ruleset.mk to 
mozilla/security/manager/Makefile.in.  I will attach both fixes.

Comment 3

[Wan-Teh Chang]

Attachment: Proposed patch 1: include security/coreconf/ruleset.mk to have CPU_TAG set properly.

Comment 4

[Wan-Teh Chang]

Attachment: Proposed patch 2: copy just the part of security/coreconf/ruleset.mk that sets CPU_TAG.

Comment 5

[Margaret Chan]

Thanks, Wan-Teh.  Do you have a bug # for that?  If not, can you update this bug
when it's in so that we know when to switch to do that.  By the way, just to let
you know.  We had rebuilt it with the NS_USE_NATIVE environment set before the
build (i.e., at the very beginning before we build mozilla), and we are seeing
the following errors:

gmake[3]: Entering directory
`/net/crumple.eng/export/nc-re/release/build/SUN-MOZ/5.7/sparc-opt/mozilla/js/src'
/usr/ccs/bin/as -o lock_SunOS.o   lock_SunOS.s
/usr/ccs/bin/as: "lock_SunOS.s", line 42: error: statement syntax
/usr/ccs/bin/as: "lock_SunOS.s", line 66: error: statement syntax
/usr/ccs/bin/as: "lock_SunOS.s", line 69: error: unknown opcode "ENTRY"
/usr/ccs/bin/as: "lock_SunOS.s", line 69: error: statement syntax
/usr/ccs/bin/as: "lock_SunOS.s", line 86: error: unknown opcode "SET_SIZE"
/usr/ccs/bin/as: "lock_SunOS.s", line 86: error: statement syntax
/usr/ccs/bin/as: "lock_SunOS.s", line 91: error: statement syntax
/usr/ccs/bin/as: "lock_SunOS.s", line 96: error: unknown opcode "ENTRY"
/usr/ccs/bin/as: "lock_SunOS.s", line 96: error: statement syntax
/usr/ccs/bin/as: "lock_SunOS.s", line 107: error: unknown opcode "SET_SIZE"
/usr/ccs/bin/as: "lock_SunOS.s", line 107: error: statement syntax
/usr/ccs/bin/as: "lock_SunOS.s", line 114: error: statement syntax
/usr/ccs/bin/as: "lock_SunOS.s", line 99: error: cannot use v8plus instructions
in a non-v8plus target binary
gmake[3]: *** [lock_SunOS.o] Error 1
gmake[3]: Leaving directory
`/net/crumple.eng/export/nc-re/release/build/SUN-MOZ/5.7/sparc-opt/mozilla/js/src'
gmake[2]: *** [install] Error 2
gmake[2]: Leaving directory
`/net/crumple.eng/export/nc-re/release/build/SUN-MOZ/5.7/sparc-opt/mozilla/js'
gmake[1]: *** [install] Error 2
gmake[1]: Leaving directory
`/net/crumple.eng/export/nc-re/release/build/SUN-MOZ/5.7/sparc-opt/mozilla'
gmake: *** [build] Error 2

So my impression is that we should only set this environment variable before we
do psm build and not before we do mozilla build.  Am I correct?  We have not
tried that yet.  Just checking. 

Just saw your proposed patches.  Thanks much.  We would try applying the first
patch and see how it goes with the PSM2 build.  Will update the bug for that. 
While on this bug, you may be able to help us on the third problem (set
LD_LIBRARY_PATH...) described below as well.  The description of that problem is
not very clear, so I am explaining it here in more details:  After I applied all
the workaround with the build, I got segmentation fault when I brought up the
browser.  This happened with our Sparc build.  Having talking with the HP
porting team, I was told that they had problem accessing secured sites because
of a misplaced library, libfreebl_hybrid_3.sl.  The library was found in
.../mozilla/dist/lib instead of .../mozilla/dist/bin where the LD_LIBRARY_PATH
was set to.  So I applied the workaround to either set the LD_LIBRARY_PATH or
set a soft link from .../mozilla/dist/bin/libfreebl_hybrid_3.sl to
.../mozilla/dist/lib/libfreebl_hybrid_3.sl.  This temporarily solved our Sparc
bringup problem.  However, the link failed on the Solaris x86 platform because
the library appeared to be platform specific (at least it did not show up on the
Solaris x86 platform).  Should we be modifying the security/manager/Makefile.in
to install this library to .../mozilla/dist/bin with some ifdefs for the
appropriate platforms.  If so, which platforms have this library?  FYI:  The
HPUX bug for the same problem is 76370.  

Comment 6

[Wan-Teh Chang]

Margaret: I can't help you with the lock_SunOS.s assembly problem in
Javascript.  (I can reproduce that problem, by the way.)  I don't think
this problem has anything to do with NS_USE_NATIVE.  Setting NS_USE_NATIVE
should not hurt the modules that don't check this make variable.

Only Solaris SPARC (32-bit) and HP-UX PA-RISC (32-bit) have
libfreebl_pure32_3.{so,sl} and libfreebl_hybrid_3.{so,sl}.  These
two libraries need to be installed in the same directory as
libnss3.{so,sl}.

Comment 7

[Margaret Chan]

Ummm...

But we had done 2 builds.  One with the workaround to hardcode NS_USE_NATIVE
inside the ../security/coreconf/SunOS.mk.  That went fine.  Then we rebuilt
using the same source.  The second time, we got rid of the change to the file.
Instead we setenv NS_USE_NATIVE 1 before the mozilla build.  This time, we saw
the assembly problem.  That's why I thought that somewhere this variable was
being checked and thought that maybe we should only set it before building psm2,
but I had not done any further search on where else it was being referenced.  I
guess we'll wait for your fix (when it gets in) before we try it again.

As for the library, it looks like all the libnss*.{so,sl} are installed in
.../mozilla/dist/lib.  And mozilla do not have the LD_LIBRARY_PATH set to that
at all.  The only library being installed in .../mozilla/bin is libnssckbi.so.
So do they (libnss*.{so,sl}) all need to be in the same place.  Or just the two
which you have mentioned?  

Comment 8

[Jim Dunn]

wtc, 
Why does HP & Solaris build ibfreebl_<blah> as shared objects/libs
instead of as .a's like Linux?  Is there any way to fix this so that
it will be built like Linux?

2ndly in HP's case, a 'hybrid' is built and I think it builds it as
DA2.0.  Will this be compatible on older machines running HP11?
Especially since the rest of psm & mozilla is built with
DAportable or DA1.0.

Comment 9

[Jim Dunn]

adding myself to cc

Comment 10

[Wan-Teh Chang]

Jim,

On 32-bit Solaris SPARC and 32-bit HP-UX PA-RISC, you have
libfreebl_pure32_3.{so,sl} and libfreebl_hybrid_3.{so,sl},
in addition to libfreebl.a.  One of these shared libraries
is loaded dynamically when NSS is initialized depending on
the processor architecture.

Take HP-UX PA-RISC for example.  libfreebl_hybrid_3.sl is
intentionally built as DA2.0 and it is only loaded when you
are running on PA-RISC 2.0 architecture.  If you are running
on the older PA-RISC 1.1 architecture, libfreebl_pure32_3.sl
is loaded instead.  Similar things happen on Solaris SPARC,
where it's SPARC v8 vs. SPARC v8plus.

This is why both of these libfreebl_<blah> shared libraries
need to be installed.

Comment 11

[Jim Dunn]

Ok, I agree from a pure security standpoint this is probably the right thing
to do.  But from a mozilla standpoint... it stinks.  If we only ship one
binary, then we have to ship 2 libfreebl's.  Since we are always building
with DA1.0 or portable anyway, I would rather just ship the pure32 shrlib.

So if we only intend (from a mozilla standpoint) to ship one shrlib...
it probably makes sense to just build this whole thing with DAportable,
build a .a and just link it in (one less shrlib, less load time...)

However to do this, I am going to hack the hell out of your makefiles
(I nor u want to do this).

And if we do leave it as a .sl then we really need to NSINSTALL this
in the $(DIST)/bin/components dir and then do a shl_load from there.
This requires a source code change in the loader.c routine.

Unless I am missing something, any change that works for mozilla 
has to be be ifdef'd for mozilla_client so as to not mess up the
other places where this is used.  

Given these 2 choices (just build .a and link with that, or install into
dist/bin/components and shl_load from there) which would you recommend

Comment 12

[Wan-Teh Chang]

Jim: you must use the libfreebl_<blah> shared libraries.  They
can be installed in any directory on the app's LD_LIBRARY_PATH (or
SHLIB_PATH for HP-UX).  I would install them in the same
directory as libnspr4.{so,sl}.

Margaret: please ignore my earlier statement regarding libnss3.{so,sl}.
Mozilla is not using libnss3.{so,sl}.

Comment 13

[Margaret Chan]

Attachment: Proposed patch for loading the libfreebl_<blah> shared libraries.

Comment 14

[Margaret Chan]

Based on Wan-Teh and Jim's comments above, it looks like that we'll have to
include both libraries.  Is .../mozilla/dist/bin a good place for them since
libnspr4.{so,sl} is installed over there and the shared library path is
definitely set over there as well.  Attached a proposed patch above.  Haven't
really tested it yet.  Just want to post it out to solicit comments/feedback.

Comment 15

[Wan-Teh Chang]

I checked in my fix for mozilla/configure.in which should make it
suffice to set NS_USE_NATIVE to 1 in your environment.

Explanation of my fix:

When MOZILLA_CLIENT is set to 1 (which PSM does when it builds NSS),
NS_USE_NATIVE must be set to 1 if you are using the Solaris compiler
(cc).  However, Mozilla's configure script was generating a
mozilla/config/autoconf.mk file that unset NS_USE_NATIVE, undoing
what was set in the environment.  My change to mozilla/configure.in
caused it to generate a mozilla/config/autoconf.mk that sets
NS_USE_NATIVE to 1 if the Solaris compiler is used.

Comment 16

[Margaret Chan]

Our RE has done a build with Wan-Teh's proposed patch 1.  The build went fine
but I was not able to verify the PSM functionality because of some bringup
problems that we were seeing.  Not sure if it had any relationship with the
patch or not.  Will post update when we get more information.

Comment 17

[Margaret Chan]

Just want to post an update so that the others are aware.  With wtc's checkin
(thanks!), the assembly problem in js/src comes back.  The assembly code
apparently has never been picked up (which in fact should be).  A checkin by cls
(added assembly flag for Solaris) resolved the assembly build problem for
Solaris-Sparc.  He will also check in another fix later on to exclude
Solaris-i86.  So before then, Solaris-i86 will still have the same assembly
problem.

Comment 18

[Jim Dunn]

Attachment: updated patch for both HP & Solaris

Comment 19

[Jim Dunn]

I have updated the patch to incorporate WTC's, Margaret's and
Shannon's diffs.  I have tested it on HP & Linux.

NOTES:
-the +='s in the sub Makefile.ins is a requirement on COMPONENT
 shared libraries.  If you lookin mozilla/config/rules.mk you
 will see that there is special code based on IS_COMPONENT=1
 and it sets EXTRA_DSO_LDOPTS, so you have to += this variable
 if you have already includes rules.mk

I am looking for review and approval so that I can check this in.
wtc?  cls?

Comment 20

[Jim Dunn]

Attachment: update security/manager/Makefile.in (see 76379)

Comment 21

[Margaret Chan]

From now on, please follow the progress of bug 76370.  That bug is opened for
the HPUX platform, which has similar problem as Solaris.  I will update this bug
if I see any Solaris specific issue.  I am leaving this bug open until Jim's
patch is in.

BTW, I have just noticed a typo in the build steps which I have posted earlier:

6. add to .mozconfig the line:  ac_add_options --enable_modules=psm2
                                               ^^^^^^^^^^^^^^^^^^^^^
should be 

6. add to .mozconfig the line:  ac_add_options --enable-modules=psm2
                                               ^^^^^^^^^^^^^^^^^^^^^

Comment 22

[Jim Dunn]

I checked in the fix, I think this bug can be closed as a dup for
http://bugzilla.mozilla.org/show_bug.cgi?id=76370

Comment 23

[Margaret Chan]

I've verified the fix on my Solaris debug build.  Marking it as a dup of 76370.

*** This bug has been marked as a duplicate of 76370 ***

Comment 24

[John Unruh]

Verified.