Closed Bug 172651 Opened 22 years ago Closed 20 years ago

Can't cross compile NSS for same OS but different CPU type

Categories

(NSS :: Libraries, defect, P3)

All
Linux
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: nelson, Assigned: wtc)

References

Details

(Whiteboard: [minimo] 1 day)

Attachments

(2 files, 7 obsolete files)

This is a continuation of bug 104541.  In that bug, Wan-Teh Chang wrote:

I think we still have difficulty cross-compiling for a different CPU 
architecture on the same OS, in particular, cross-compiling for Linux ARM in
Linux x86 as Mark Crichton pointed out in comment #6.

OS_TARGET specifies the target OS, which is the same (Linux) in this case.  
We need a make variable for specifying the target CPU architecture.  
It seems that we can just use the existing CPU_ARCH for this purpose, 
that is, specifically define CPU_ARCH to mean the target, not the host, 
CPU architecture, and allow CPU_ARCH to be overridden for cross-compilation.
this is going to be a blocker for cross-compiling minimo under ix86-linux for
arm-linux.
Whiteboard: [minimo]
see the comments dated April 20th on
http://playstation2-linux.com/projects/mozilla-ps2 - he says he has PSM cross
compiling! we need to get hold of ppietro@users.playstation2-linux.com (he
doesn't seem to have a bugzilla account) and see if we can get him to contribute
back his work.
(I'm sending him an e-mail right now)
Hiya!

Okay - I've joined Bugzilla. (^_^)

Unfortunately, I *haven't* figured out how to cross compile.  What I figured out
was how to compile NSS natively for the mips architecture of the PlayStation 2.

This bug affects Mozilla for PlayStation 2 Linux as well, since we generally
cross compile for Linux mips in Linux x86.  

Cheers,
Paul
As of NSS 3.4, NSS could be cross built on Windows X86 for WinCE on ARM CPUs.
That build included a limited subset of the commands.

I believe that no longer works for two reasons:

a) NSS now builds a tool named shlibsign, and runs that tool during the build.
Obviously, if the tool is built for a different CPU than the one doing the
building, shlibsign won't run on the build system.

b) I don't think it is possible to build NSPR on the trunk for WinCE.  NSPR for
WinCE was developed on a branch.  I am not aware that that branch was ever
merged to the trunk.  NSS depends on NSPR enhancements in the most recent trunk
version of NSPR.  

Marking P3. Controversy expected.
Priority: -- → P3
this goes with the following patch, which is a new config file (LinuxARM.mk)
for security/coreconf.
okay, this patch and corresponding LinuxARM.mk file are not exactly pretty. 
suggestions and reviews are definitely welcome.

some issues:

  o  the existing Linux-XXX.mk files differentiate the Linux version being 
     targeted.  i only created one LinuxARM.mk file, which assumes Linux 2.1 or 
     better.

  o  my changes to security/manager/Makefile.in only works when cross-compiling
     minimo.  i think that is probably wrong.  it seems like it needs to look
     at OS_ARCH and TARGET_CPU, which are set by the toplevel configure script.

  o  as nelson pointed out, in a cross-compilation environment we cannot run
     shlibsign, so i hacked around this problem by modifying sigh.sh to not
     do anything when the target is LinuxARM.  i'm not sure i like this, and
     i may just modify the PSM makefile to not execute shlibsign.  i'm leaning
     toward this latter change since i had to anyways modify PSM's makefile to
     have it not try to install the .chk files.

so, there is still some more work to do.  i just wanted to get this into this
bug so that others can take a look.  please let me know if you have any
suggestions!! :-)
shouldn't the signing app be compiled to the host platform? it's not used by the
target, right?
timeless: sure, that would be nice.  but, that requires compiling NSS for both
the host and target platforms.  that sounds like two passes over NSS, which i
figure means more significant build changes.
Blocks: 215636
Whiteboard: [minimo] → [minimo] 1 day
I just wanted to note that a general purpose (though brutish) cross-compiling
solution was attached to the bug 104541.  The shlibgen problem still needs to be
resolved though and it would require building NSS for both the host & target
(just as we do for xpidl).  And to overstate the not-so-obvious, that would
require building NSPR & dbm for both host & target as well.
wan-teh, where are we with this patch?  I would like to see it land on the trunk
if possible.
Attached patch wtc's patch v1 (use CPU_TARGET) (obsolete) — Splinter Review
Darin, Doug, please try this patch.

You should back out your patch before applying
this patch.  Also remember to remove LinuxARM.mk.
i'm testing this patch now...
the patch required some small modifications to build (nothing major).  however,
the security component does not seem to be working in my build.  i'm not sure if
it has anything to do with this patch (since i have not tested my patch on the
trunk in ages):

a necko socket transport log reveals the following error while trying to write
out the HTTP request to https://www.verisign.com

16386[112540]:   calling PR_Write [count=297]
16386[112540]:   PR_Write returned [n=-1]
16386[112540]: ErrorAccordingToNSPR [in=-8181 out=80004005]

NSPR does not appear to define the error -8181

Anyone have any idea what that means?  Meanwhile I'm going to revert to the
LinuxARM.mk patch just to rule out the possibility of this patch being the culprit.
i think -8181 corresponds to SEC_ERROR_EXPIRED_CERTIFICATE.  a ssl trace shows:

686: SSL3[1654504]: peer certificate is no good: error=-8181

:-(
OK, the same problem happens with either patch.  This looks like some kind of
unrelated NSS regression.
Attached patch wtc's patch v1.1 (obsolete) — Splinter Review
this is wtc's patch w/ a typo fix and the freebl makefile changes he mentioned
over AIM.  it compiles, but as i've said.. it doesn't seem to work (even though
the same time loads fine under today's nightly build of x86-linux firefox).
Attachment #143678 - Attachment is obsolete: true
s/same time/same site/  ... not to be confused with Lotus SameTime :-P
> i think -8181 corresponds to SEC_ERROR_EXPIRED_CERTIFICATE.  a ssl trace shows:
> 686: SSL3[1654504]: peer certificate is no good: error=-8181

ok, so it turns out that it's helpful to have a properly configured clock on the
client machine :-/
Comment on attachment 143699 [details] [diff] [review]
wtc's patch v1.1

Re: comment 18

Darin, what's the typo fix?  I can't find it in this patch.
wtc:

+ifneq SKIP_CHK

should have been:

+ifndef SKIP_CHK

this is in the PSM makefile.
Comment on attachment 143699 [details] [diff] [review]
wtc's patch v1.1

Your patch still says:

>+ifneq SKIP_CHK
> 	$(INSTALL) -m 644 $(DIST)/lib/$(SOFTOKEN3_CHK) $(DIST)/bin
>+endif

This is why I was confused :-)
Attachment #136508 - Attachment is obsolete: true
Attachment #136509 - Attachment is obsolete: true
Attachment #143699 - Attachment is obsolete: true
Splitting up the patch so that the hardcoded changes inside NSS can land as
soon as possible.  On the mozilla side, we need to acknowledge that minimo
isn't our only cross-compiling target and use the generic method for
cross-compiling NSS.
Attachment #144003 - Flags: superreview?(wchang0222)
Attachment #144003 - Flags: review?(darin)
Attachment #144004 - Flags: superreview?(bryner)
Attachment #144004 - Flags: review?(darin)
Flags: blocking1.7b?
Attachment #144003 - Flags: review?(darin) → review+
Attachment #144004 - Flags: review?(darin) → review+
Comment on attachment 144004 [details] [diff] [review]
moz changes for cross-compiling nss

cls, thanks for your NSS and Mozilla patches.

Your changes to security/manager/Makefile.in have
two problems.

1. I'd like you to remove the following two lines:

>+DEFAULT_GMAKE_FLAGS += NSINSTALL="$(NSINSTALL)"
>+DEFAULT_GMAKE_FLAGS += MKDEPEND="$(MKDEPEND)"

NSS's coreconf system uses the make variable NATIVE_CC
to compile 'nsinstall' if NATIVE_CC is defined, so you
don't need to override NSINSTALL.

'mkdepend' is not used by NSS.	Even if it were, it
would have been compiled with NATIVE_CC.

2. The following code makes most of the changes to NSS
(in the other patch) unnecessary.

>+ifdef CROSS_COMPILE
>+DEFAULT_GMAKE_FLAGS += \
>+	CC="$(CC)" \
>+	CCC="$(CXX)" \
>+	LINK="$(LD)" \
>+	AS="$(AS)" \
>+	AR='$(AR) $(AR_FLAGS:$@=$$@)' \
>+	RANLIB="$(RANLIB)" \
>+	RC="$(RC) $(RCFLAGS)" \
>+	OS_ARCH="$(OS_ARCH)" \
>+	TARGET_CPU="$(TARGET_CPU)" \
>+	$(NULL)
>+SKIP_CHK=1
>+endif

So, if you want to check in the above change, we should
not check in most of the NSS changes.  (You'd also need
to add
	NATIVE_CC="$(HOST_CC)" \
to take care of 'nsinstall'.  I am fine with changing
NSS to use HOST_CC.)

The above change also has a bug: the line before $(NULL)
should read:
	CPU_TARGET="$(TARGET_CPU)" \
Attachment #144004 - Flags: superreview?(bryner) → superreview-
Comment on attachment 144003 [details] [diff] [review]
NSS cross-arch cross-compile changes

The changes in this patch are basically fine.

There is one change that we don't want to take:

>Index: security/coreconf/Makefile
...
>+ifndef MOZILLA_CLIENT
> DIRS		= nsinstall
>+endif

As I explained in the comment for the other patch,
if you define NATIVE_CC, NSS will compile its own
'nsinstall' with $(NATIVE_CC), so it is not necessary
to make the above change.

This change is good:

>Index: security/coreconf/rules.mk
...
>-ifeq (,$(filter-out WIN%,$(OS_TARGET)))
>+ifeq (,$(filter-out _WIN%,$(NS_USE_GCC)_$(OS_TARGET)))
> 	$(AR) $(subst /,\\,$(OBJS))
> else
> 	$(AR) $(OBJS)

All the other changes will be bypassed if you check in
your "brute force" NSS cross compile changes in the other
patch, so I am not sure if we need to check in the other
changes.

I am marking this patch review+ on the condition that
the change to security/coreconf/Makefile be removed.
Attachment #144003 - Flags: superreview?(wchang0222) → superreview+
(In reply to comment #26)

> 1. I'd like you to remove the following two lines:
> 
> >+DEFAULT_GMAKE_FLAGS += NSINSTALL="$(NSINSTALL)"
> >+DEFAULT_GMAKE_FLAGS += MKDEPEND="$(MKDEPEND)"
> 
> NSS's coreconf system uses the make variable NATIVE_CC
> to compile 'nsinstall' if NATIVE_CC is defined, so you
> don't need to override NSINSTALL.

When nsinstall is built, even in the non-cross-compiling case, coreconf's
nsinstall overwrites the version installed into $(DIST)/bin by mozilla.  We
don't want that.

> 2. The following code makes most of the changes to NSS
> (in the other patch) unnecessary.

Well, yes and no.  The brute force changes make the hardcoded values in NSS
unnecessary when building Mozilla.  However, you'd still want them if building
NSS standalone...or maybe you just require them to override the same variables.

> So, if you want to check in the above change, we should
> not check in most of the NSS changes.  (You'd also need
> to add
> 	NATIVE_CC="$(HOST_CC)" \
> to take care of 'nsinstall'.  I am fine with changing
> NSS to use HOST_CC.)

I'd rather not build nsinstall but I can add that so that eventually we can
build shlibgen.
Attached patch moz changes (take 2) (obsolete) — Splinter Review
I removed the NSINSTALL setting and I hit another issue that I had forgotten
about.	Like mozilla, NSS assumes nsinstall is in the path when building for
target win32.  This isn't the case when cross-compiling so the build fails.
Attachment #144004 - Attachment is obsolete: true
Attachment #144083 - Flags: superreview?(wchang0222)
cls, thank you for explaining the nsinstall overwrite issue.
In this patch I propose a different solution: NSS doesn't
install its 'nsinstall' in mozilla's dist/bin directory.
I do that by setting the INSTALL make variable to 'true'
to make it a no-op.  (Should I set it to '/bin/true'?)

This patch doesn't include the real cross-arch cross-compile
changes.  I understand that they are useful for cross-compiling
NSS *standalone* for Linux/arm, but I don't know of anyone who
does that, so I don't want to check in code that won't get used.

Please review and test this patch.
Comment on attachment 144187 [details] [diff] [review]
Minimal NSS changes

When testing this patch, you should change "CPU_TARGET"
to "CPU_ARCH" in the "moz changes" patch.
Comment on attachment 144083 [details] [diff] [review]
moz changes (take 2)

In this patch, please try overwriting NSINSTALL only
if CROSS_COMPILE is defined.
Flags: blocking1.7b?
Flags: blocking1.7b-
Flags: blocking1.7?
Attachment #144003 - Attachment is obsolete: true
Attachment #144083 - Attachment is obsolete: true
Attachment #144083 - Flags: superreview?(wchang0222)
these last two patches seem to work great!  does that mean we are ready to go? 
you guys want to sign off on the patches so we can get them checked in?  thx!
Attachment #144187 - Flags: approval1.7?
Attachment #144281 - Flags: approval1.7?
Attachment #144187 - Flags: superreview+
Comment on attachment 144187 [details] [diff] [review]
Minimal NSS changes

a=chofmann for 1.7
Attachment #144187 - Flags: approval1.7? → approval1.7+
Comment on attachment 144281 [details] [diff] [review]
moz changes (take 3)

a=chofmann for 1.7
Attachment #144281 - Flags: approval1.7? → approval1.7+
Comment on attachment 144187 [details] [diff] [review]
Minimal NSS changes

I've checked in this patch on the NSS trunk (NSS 3.10),
NSS_3_9_BRANCH (NSS 3.9.1), and NSS_CLIENT_TAG (Mozilla
1.7 final).
Comment on attachment 144281 [details] [diff] [review]
moz changes (take 3)

>Index: security/manager/Makefile.in
...
>+ifdef CROSS_COMPILE
>+DEFAULT_GMAKE_FLAGS += \
>+	NSINSTALL="$(NSINSTALL)" \
>+	NATIVE_CC="$(HOST_CC)" \
>+	CC="$(CC)" \
>+	CCC="$(CXX)" \
>+	LINK="$(LD)" \
>+	AS="$(AS)" \
>+	AR='$(AR) $(AR_FLAGS:$@=$$@)' \
>+	RANLIB="$(RANLIB)" \
>+	RC="$(RC) $(RCFLAGS)" \
>+	OS_ARCH="$(OS_ARCH)" \

It may be more correct to override OS_TARGET
instead of OS_ARCH.  I am not sure though.
In coreconf, OS_TARGET defaults to OS_ARCH with the exception of compiling for
WIN95 or WIN16 so I'll have to fix that for cross-mingw builds.  Currently, when
cross-compiling, Mozilla's OS_TARGET doesn't match what NSS would expect.
(In reply to comment #39)
> In coreconf, OS_TARGET defaults to OS_ARCH with the exception of compiling for
> WIN95 or WIN16 so I'll have to fix that for cross-mingw builds.  Currently, when
> cross-compiling, Mozilla's OS_TARGET doesn't match what NSS would expect.

Taking a closer look, we're good to go for cross-compiling to win32.  We already
add OS_TARGET=WIN95 to DEFAULT_GMAKE_FLAGS when building NSS for win32.
Checking in security/manager/Makefile.in;
/cvsroot/mozilla/security/manager/Makefile.in,v  <--  Makefile.in
new revision: 1.53; previous revision: 1.52

Status: NEW → RESOLVED
Closed: 20 years ago
Flags: blocking1.7?
Resolution: --- → FIXED
Comment on attachment 144281 [details] [diff] [review]
moz changes (take 3)

>+	AR='$(AR) $(AR_FLAGS:$@=$$@)' \
I'm trying to cross-compile but the $@ always disappears from AR_FLAGS when I try... is that because it's a macro rather than a variable?
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: