172651 - Can't cross compile NSS for same OS but different CPU type

Status: RESOLVED FIXED

(NSS :: Libraries, defect, P3)

Description

[Nelson Bolyard (seldom reads bugmail)]

This is a continuation of bug 104541.  In that bug, Wan-Teh Chang wrote:

I think we still have difficulty cross-compiling for a different CPU 
architecture on the same OS, in particular, cross-compiling for Linux ARM in
Linux x86 as Mark Crichton pointed out in comment #6.

OS_TARGET specifies the target OS, which is the same (Linux) in this case.  
We need a make variable for specifying the target CPU architecture.  
It seems that we can just use the existing CPU_ARCH for this purpose, 
that is, specifically define CPU_ARCH to mean the target, not the host, 
CPU architecture, and allow CPU_ARCH to be overridden for cross-compilation.

Comment 1

[Darin Fisher]

this is going to be a blocker for cross-compiling minimo under ix86-linux for
arm-linux.

Comment 2

[Alec Flett]

see the comments dated April 20th on
http://playstation2-linux.com/projects/mozilla-ps2 - he says he has PSM cross
compiling! we need to get hold of [e-mail] (he
doesn't seem to have a bugzilla account) and see if we can get him to contribute
back his work.

Comment 3

[Alec Flett]

(I'm sending him an e-mail right now)

Comment 4

[Paul Pietromonaco]

Hiya!

Okay - I've joined Bugzilla. (^_^)

Unfortunately, I *haven't* figured out how to cross compile.  What I figured out
was how to compile NSS natively for the mips architecture of the PlayStation 2.

This bug affects Mozilla for PlayStation 2 Linux as well, since we generally
cross compile for Linux mips in Linux x86.  

Cheers,
Paul

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

As of NSS 3.4, NSS could be cross built on Windows X86 for WinCE on ARM CPUs.
That build included a limited subset of the commands.

I believe that no longer works for two reasons:

a) NSS now builds a tool named shlibsign, and runs that tool during the build.
Obviously, if the tool is built for a different CPU than the one doing the
building, shlibsign won't run on the build system.

b) I don't think it is possible to build NSPR on the trunk for WinCE.  NSPR for
WinCE was developed on a branch.  I am not aware that that branch was ever
merged to the trunk.  NSS depends on NSPR enhancements in the most recent trunk
version of NSPR.  

Marking P3. Controversy expected.

Comment 6

[Darin Fisher]

Attachment: Working patch to enable cross-compiling from Linux-x86 to Linux-ARM (includes PSM Makefile.in changes)

this goes with the following patch, which is a new config file (LinuxARM.mk)
for security/coreconf.

Comment 7

[Darin Fisher]

Attachment: LinuxARM.mk (corresponding file for security/coreconf)

Comment 8

[Darin Fisher]

okay, this patch and corresponding LinuxARM.mk file are not exactly pretty. 
suggestions and reviews are definitely welcome.

some issues:

  o  the existing Linux-XXX.mk files differentiate the Linux version being 
     targeted.  i only created one LinuxARM.mk file, which assumes Linux 2.1 or 
     better.

  o  my changes to security/manager/Makefile.in only works when cross-compiling
     minimo.  i think that is probably wrong.  it seems like it needs to look
     at OS_ARCH and TARGET_CPU, which are set by the toplevel configure script.

  o  as nelson pointed out, in a cross-compilation environment we cannot run
     shlibsign, so i hacked around this problem by modifying sigh.sh to not
     do anything when the target is LinuxARM.  i'm not sure i like this, and
     i may just modify the PSM makefile to not execute shlibsign.  i'm leaning
     toward this latter change since i had to anyways modify PSM's makefile to
     have it not try to install the .chk files.

so, there is still some more work to do.  i just wanted to get this into this
bug so that others can take a look.  please let me know if you have any
suggestions!! :-)

Comment 9

[timeless]

shouldn't the signing app be compiled to the host platform? it's not used by the
target, right?

Comment 10

[Darin Fisher]

timeless: sure, that would be nice.  but, that requires compiling NSS for both
the host and target platforms.  that sounds like two passes over NSS, which i
figure means more significant build changes.

Comment 11

[cls]

I just wanted to note that a general purpose (though brutish) cross-compiling
solution was attached to the bug 104541.  The shlibgen problem still needs to be
resolved though and it would require building NSS for both the host & target
(just as we do for xpidl).  And to overstate the not-so-obvious, that would
require building NSPR & dbm for both host & target as well.

Comment 12

[Doug Turner (:dougt)]

wan-teh, where are we with this patch?  I would like to see it land on the trunk
if possible.

Comment 13

[Wan-Teh Chang]

Attachment: wtc's patch v1 (use CPU_TARGET)

Darin, Doug, please try this patch.

You should back out your patch before applying
this patch.  Also remember to remove LinuxARM.mk.

Comment 14

[Darin Fisher]

i'm testing this patch now...

Comment 15

[Darin Fisher]

the patch required some small modifications to build (nothing major).  however,
the security component does not seem to be working in my build.  i'm not sure if
it has anything to do with this patch (since i have not tested my patch on the
trunk in ages):

a necko socket transport log reveals the following error while trying to write
out the HTTP request to https://www.verisign.com

16386[112540]:   calling PR_Write [count=297]
16386[112540]:   PR_Write returned [n=-1]
16386[112540]: ErrorAccordingToNSPR [in=-8181 out=80004005]

NSPR does not appear to define the error -8181

Anyone have any idea what that means?  Meanwhile I'm going to revert to the
LinuxARM.mk patch just to rule out the possibility of this patch being the culprit.

Comment 16

[Darin Fisher]

i think -8181 corresponds to SEC_ERROR_EXPIRED_CERTIFICATE.  a ssl trace shows:

686: SSL3[1654504]: peer certificate is no good: error=-8181

:-(

Comment 17

[Darin Fisher]

OK, the same problem happens with either patch.  This looks like some kind of
unrelated NSS regression.

Comment 18

[Darin Fisher]

Attachment: wtc's patch v1.1

this is wtc's patch w/ a typo fix and the freebl makefile changes he mentioned
over AIM.  it compiles, but as i've said.. it doesn't seem to work (even though
the same time loads fine under today's nightly build of x86-linux firefox).

Comment 19

[Darin Fisher]

s/same time/same site/  ... not to be confused with Lotus SameTime :-P

Comment 20

[Darin Fisher]

> i think -8181 corresponds to SEC_ERROR_EXPIRED_CERTIFICATE.  a ssl trace shows:
> 686: SSL3[1654504]: peer certificate is no good: error=-8181

ok, so it turns out that it's helpful to have a properly configured clock on the
client machine :-/

Comment 21

[Wan-Teh Chang]

Comment on attachment 143699 [details] [diff] [review]
wtc's patch v1.1

Re: comment 18

Darin, what's the typo fix?  I can't find it in this patch.

Comment 22

[Darin Fisher]

wtc:

+ifneq SKIP_CHK

should have been:

+ifndef SKIP_CHK

this is in the PSM makefile.

Comment 23

[Wan-Teh Chang]

Comment on attachment 143699 [details] [diff] [review]
wtc's patch v1.1

Your patch still says:

>+ifneq SKIP_CHK
> 	$(INSTALL) -m 644 $(DIST)/lib/$(SOFTOKEN3_CHK) $(DIST)/bin
>+endif

This is why I was confused :-)

Comment 24

[cls]

Attachment: NSS cross-arch cross-compile changes

Comment 25

[cls]

Attachment: moz changes for cross-compiling nss

Splitting up the patch so that the hardcoded changes inside NSS can land as
soon as possible.  On the mozilla side, we need to acknowledge that minimo
isn't our only cross-compiling target and use the generic method for
cross-compiling NSS.

Comment 26

[Wan-Teh Chang]

Comment on attachment 144004 [details] [diff] [review]
moz changes for cross-compiling nss

cls, thanks for your NSS and Mozilla patches.

Your changes to security/manager/Makefile.in have
two problems.

1. I'd like you to remove the following two lines:

>+DEFAULT_GMAKE_FLAGS += NSINSTALL="$(NSINSTALL)"
>+DEFAULT_GMAKE_FLAGS += MKDEPEND="$(MKDEPEND)"

NSS's coreconf system uses the make variable NATIVE_CC
to compile 'nsinstall' if NATIVE_CC is defined, so you
don't need to override NSINSTALL.

'mkdepend' is not used by NSS.	Even if it were, it
would have been compiled with NATIVE_CC.

2. The following code makes most of the changes to NSS
(in the other patch) unnecessary.

>+ifdef CROSS_COMPILE
>+DEFAULT_GMAKE_FLAGS += \
>+	CC="$(CC)" \
>+	CCC="$(CXX)" \
>+	LINK="$(LD)" \
>+	AS="$(AS)" \
>+	AR='$(AR) $(AR_FLAGS:$@=$$@)' \
>+	RANLIB="$(RANLIB)" \
>+	RC="$(RC) $(RCFLAGS)" \
>+	OS_ARCH="$(OS_ARCH)" \
>+	TARGET_CPU="$(TARGET_CPU)" \
>+	$(NULL)
>+SKIP_CHK=1
>+endif

So, if you want to check in the above change, we should
not check in most of the NSS changes.  (You'd also need
to add
	NATIVE_CC="$(HOST_CC)" \
to take care of 'nsinstall'.  I am fine with changing
NSS to use HOST_CC.)

The above change also has a bug: the line before $(NULL)
should read:
	CPU_TARGET="$(TARGET_CPU)" \

Comment 27

[Wan-Teh Chang]

Comment on attachment 144003 [details] [diff] [review]
NSS cross-arch cross-compile changes

The changes in this patch are basically fine.

There is one change that we don't want to take:

>Index: security/coreconf/Makefile
...
>+ifndef MOZILLA_CLIENT
> DIRS		= nsinstall
>+endif

As I explained in the comment for the other patch,
if you define NATIVE_CC, NSS will compile its own
'nsinstall' with $(NATIVE_CC), so it is not necessary
to make the above change.

This change is good:

>Index: security/coreconf/rules.mk
...
>-ifeq (,$(filter-out WIN%,$(OS_TARGET)))
>+ifeq (,$(filter-out _WIN%,$(NS_USE_GCC)_$(OS_TARGET)))
> 	$(AR) $(subst /,\\,$(OBJS))
> else
> 	$(AR) $(OBJS)

All the other changes will be bypassed if you check in
your "brute force" NSS cross compile changes in the other
patch, so I am not sure if we need to check in the other
changes.

I am marking this patch review+ on the condition that
the change to security/coreconf/Makefile be removed.

Comment 28

[cls]

(In reply to comment #26)

> 1. I'd like you to remove the following two lines:
> 
> >+DEFAULT_GMAKE_FLAGS += NSINSTALL="$(NSINSTALL)"
> >+DEFAULT_GMAKE_FLAGS += MKDEPEND="$(MKDEPEND)"
> 
> NSS's coreconf system uses the make variable NATIVE_CC
> to compile 'nsinstall' if NATIVE_CC is defined, so you
> don't need to override NSINSTALL.

When nsinstall is built, even in the non-cross-compiling case, coreconf's
nsinstall overwrites the version installed into $(DIST)/bin by mozilla.  We
don't want that.

> 2. The following code makes most of the changes to NSS
> (in the other patch) unnecessary.

Well, yes and no.  The brute force changes make the hardcoded values in NSS
unnecessary when building Mozilla.  However, you'd still want them if building
NSS standalone...or maybe you just require them to override the same variables.

> So, if you want to check in the above change, we should
> not check in most of the NSS changes.  (You'd also need
> to add
> 	NATIVE_CC="$(HOST_CC)" \
> to take care of 'nsinstall'.  I am fine with changing
> NSS to use HOST_CC.)

I'd rather not build nsinstall but I can add that so that eventually we can
build shlibgen.

Comment 29

[cls]

Attachment: moz changes (take 2)

I removed the NSINSTALL setting and I hit another issue that I had forgotten
about.	Like mozilla, NSS assumes nsinstall is in the path when building for
target win32.  This isn't the case when cross-compiling so the build fails.

Comment 30

[Wan-Teh Chang]

Attachment: Minimal NSS changes

cls, thank you for explaining the nsinstall overwrite issue.
In this patch I propose a different solution: NSS doesn't
install its 'nsinstall' in mozilla's dist/bin directory.
I do that by setting the INSTALL make variable to 'true'
to make it a no-op.  (Should I set it to '/bin/true'?)

This patch doesn't include the real cross-arch cross-compile
changes.  I understand that they are useful for cross-compiling
NSS *standalone* for Linux/arm, but I don't know of anyone who
does that, so I don't want to check in code that won't get used.

Please review and test this patch.

Comment 31

[Wan-Teh Chang]

Comment on attachment 144187 [details] [diff] [review]
Minimal NSS changes

When testing this patch, you should change "CPU_TARGET"
to "CPU_ARCH" in the "moz changes" patch.

Comment 32

[Wan-Teh Chang]

Comment on attachment 144083 [details] [diff] [review]
moz changes (take 2)

In this patch, please try overwriting NSINSTALL only
if CROSS_COMPILE is defined.

Comment 33

[cls]

Attachment: moz changes (take 3)

Comment 34

[Darin Fisher]

these last two patches seem to work great!  does that mean we are ready to go? 
you guys want to sign off on the patches so we can get them checked in?  thx!

Comment 35

[chris hofmann]

Comment on attachment 144187 [details] [diff] [review]
Minimal NSS changes

a=chofmann for 1.7

Comment 36

[chris hofmann]

Comment on attachment 144281 [details] [diff] [review]
moz changes (take 3)

a=chofmann for 1.7

Comment 37

[Wan-Teh Chang]

Comment on attachment 144187 [details] [diff] [review]
Minimal NSS changes

I've checked in this patch on the NSS trunk (NSS 3.10),
NSS_3_9_BRANCH (NSS 3.9.1), and NSS_CLIENT_TAG (Mozilla
1.7 final).

Comment 38

[Wan-Teh Chang]

Comment on attachment 144281 [details] [diff] [review]
moz changes (take 3)

>Index: security/manager/Makefile.in
...
>+ifdef CROSS_COMPILE
>+DEFAULT_GMAKE_FLAGS += \
>+	NSINSTALL="$(NSINSTALL)" \
>+	NATIVE_CC="$(HOST_CC)" \
>+	CC="$(CC)" \
>+	CCC="$(CXX)" \
>+	LINK="$(LD)" \
>+	AS="$(AS)" \
>+	AR='$(AR) $(AR_FLAGS:$@=$$@)' \
>+	RANLIB="$(RANLIB)" \
>+	RC="$(RC) $(RCFLAGS)" \
>+	OS_ARCH="$(OS_ARCH)" \

It may be more correct to override OS_TARGET
instead of OS_ARCH.  I am not sure though.

Comment 39

[cls]

In coreconf, OS_TARGET defaults to OS_ARCH with the exception of compiling for
WIN95 or WIN16 so I'll have to fix that for cross-mingw builds.  Currently, when
cross-compiling, Mozilla's OS_TARGET doesn't match what NSS would expect.

Comment 40

[cls]

(In reply to comment #39)
> In coreconf, OS_TARGET defaults to OS_ARCH with the exception of compiling for
> WIN95 or WIN16 so I'll have to fix that for cross-mingw builds.  Currently, when
> cross-compiling, Mozilla's OS_TARGET doesn't match what NSS would expect.

Taking a closer look, we're good to go for cross-compiling to win32.  We already
add OS_TARGET=WIN95 to DEFAULT_GMAKE_FLAGS when building NSS for win32.

Comment 41

[cls]

Checking in security/manager/Makefile.in;
/cvsroot/mozilla/security/manager/Makefile.in,v  <--  Makefile.in
new revision: 1.53; previous revision: 1.52

Comment 42

[[e-mail]]

Comment on attachment 144281 [details] [diff] [review]
moz changes (take 3)

>+	AR='$(AR) $(AR_FLAGS:$@=$$@)' \
I'm trying to cross-compile but the $@ always disappears from AR_FLAGS when I try... is that because it's a macro rather than a variable?