462390 - Windows Mobile cross-compilation shouldn't require intermediate tools.

Status: RESOLVED INVALID

(Firefox Build System :: General, defect)

Description

[Brian Crowder]

Current, Windows Mobile is cross-compiled by defining fake "gcc" tools which are referenced by the build system, instead of the alternative cl.exe and link.exe provided by the Microsoft WinCE SDKs.  We should fix this, as it will simplify mobile development greatly.

Comment 1

[Brian Crowder]

Attachment: stuck

I've spent way too long on this, but I seem to have really genuinely gotten stuck.  Here's the patch I am currently using.

Comment 2

[Brian Crowder]

Attachment: the mozconfig file to match the patch

Comment 3

[Brian Crowder]

Running with this patch and this mozconfig, the following error occurs:

C:/PROGRA~1/MICROS~1.0/VC/CE/BIN/X86_ARM/link.exe -DLL -DEBUG -DEBUGTYPE:CV -OUT:"c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/nssutil3.dll" -DEF:c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/nssutil.def -MAP  c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/quickder.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secdig.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/derdec.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/derenc.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/dersubr.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/dertime.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/nssb64d.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/nssb64e.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/nssrwlk.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/nssilock.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/oidstring.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/portreg.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secalgid.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secasn1d.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secasn1e.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secasn1u.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secitem.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secoid.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/sectime.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secport.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/secinit.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/templates.obj c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/utf8.obj  -VERBOSE corelibc.lib winsock.lib coredll.lib -LIBPATH:C:/PROGRA~1/WI66F8~1/POCKETPC/LIB/ARMV4I -LIBPATH:c:/Users/crowder/mobilla/build/wince/shunt//build/vs9 mozce_shunt.lib -NODEFAULTLIB:LIBCMT c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/dist/lib/plc4.lib c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/dist/lib/plds4.lib c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/dist/lib/nspr4.lib    c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/nssutil.res
Microsoft (R) Incremental Linker Version 9.00.21022.08
Copyright (C) Microsoft Corporation.  All rights reserved.

C:/PROGRA~1/MICROS~1.0/VC/CE/BIN/X86_ARM/link.exe
Processed /NODEFAULTLIB:LIBCMT

Starting pass 1
C:/PROGRA~1/MICROS~1.0/VC/CE/BIN/X86_ARM/link.exe : fatal error LNK1107: invalid or corrupt file: cannot read at 0x290
make[7]: *** [c:/Users/crowder/mobilla/objdir-wm6-dbg-opt/xulrunner/nss/nssutil/nssutil3.dll] Error 83
make[7]: Leaving directory `/c/Users/crowder/mobilla/security/nss/lib/util'

And so on.  If I enter the above command precisely, using a separate command prompt, it links successfully.  I am not able to accurately discern why the link.exe tool behaves so differently within make, as opposed to run by hand.

Comment 4

[John Wolfe (:wolfe)]

Some further investigating by me showedthe following:

Theory 1: Perhaps the Environment Variables were getting corrupted by the 6th and 7th invocation of make.

Testing Theory 1: Dumping out the environment variables from the make, duplicating those environment variables in a bash shell directly (/bin/bash ==> c:\mozilla-build\msys\bin\bash.exe), then running the make command directly from the command line.  This sequence worked directly from the command line.  SO, it appears as if the bash shell itself is not directly responsible for the LNK1107 error.

Theory 2: The LIBPATH c:/Users/crowder/mobilla/build/wince/shunt//build/vs9 has two forward slashes in it - could that be causing a problem?

Testing Theory 2: By modifying the MOZCONFIG file, I was able to get rid of the two forward slashes, and caused the LIBPATH to be c:/Users/crowder/mobilla/build/wince/shunt/build/vs9.  This did not fix the LNK1107 link error.

It appears as if the two slashes in the LIBPATH does not cause the LNK1107 error.

Theory 3: The libs included in the direct command line do not match those included by the WinCE Shunt Tool arm-wince-link.exe.

Testing Theory 3: I fiddled with the rules.mk file to ensure that the command line executed by this build matched the command line created by the arm-wince-link.exe tool.  I still saw the LNK1107 error upon linking.

It appears as if the exact libraries explicitly specified does not effect the LNK1107 error.


Theory 4: No linking is ever successfully performed by the build for WinCE.

Testing Theory 4: I redirected all output from a "make -f client.mk build" command line, and searched the output for evidence of another, successful linking operation -- I found no other call to link.exe

It appears as if the calls to link.exe are failing when called from the make (th or 7th level in), but succeeding with the same command line when executed directly from the bash shell.

Theory 5: The bash shell is somehow running out of file handles.

Testing Theory 5: I have no clear way to test this theory.  I could look at the bash.exe source code and try to see how many file handles each bash program allows (and uses) - but I stopped work here.  I do not know where to get the sources for the version of bash.exe included with mozilla-build.

Upshot of all this testing?  I could not figure out what is causing the arm-wince-link.exe to succeed, and the make-called link.exe to fail.

Comment 5

[Brad Lassey [:blassey] (use needinfo?)]

Attachment: successfully builds up to netwerk

With this patch I build up to this point:

C:/PROGRA~1/MICROS~1.0/VC/CE/BIN/X86_ARM/cl.exe -FonsInputStreamChannel.obj -c
-DMOZILLA_INTERNAL_API -D_IMPL_NS_COM -DEXPORT_XPT_API -DEXPORT_XPTC_API -D_IMPL
_NS_COM_OBSOLETE -D_IMPL_NS_GFX -D_IMPL_NS_WIDGET -DIMPL_XREAPI -DIMPL_NS_NET -D
IMPL_THEBES  -DZLIB_INTERNAL -DOSTYPE=\"WINCE\" -DOSARCH=WINCE -DIMPL_NS_NET  -I
/c/hg/mozilla-central/netwerk/base/src -I. -I../../../dist/include/xpcom -I../..
/../dist/include/string -I../../../dist/include/mimetype -I../../../dist/include
/uconv -I../../../dist/include/locale -I../../../dist/include/pref -I../../../di
st/include   -I../../../dist/include/necko -Ic:/hg/mozilla-central/objdir-wm6-db
g-cmd-cl/xulrunner/dist/include/nspr     -Ic:/hg/mozilla-central/objdir-wm6-dbg-
cmd-cl/xulrunner/dist/sdk/include       -GR- -X -I/c/hg/mozilla-central/objdir-w
m6-dbg-cmd-cl/xulrunner/dist/include/shunt -Ic:/hg/mozilla-central/build/wince/s
hunt//include/ -IC:/PROGRA~1/WINDOW~4/POCKETPC/Include/Armv4i -IC:/PROGRA~1/MICR
OS~1.0/VC/CE/Include -FI mozce_shunt.h -DARM -D_ARM_ -DMOZCE_STATIC_BUILD -DUNIC
ODE -D_UNICODE_ -DWINCE -D_WIN32_WCE=0x502 -DUNDER_CE -DWIN32_PLATFORM_WFSP -D_W
INDOWS -DNO_ERRNO -Zc:wchar_t- -GS- -GR- -fp:fast -TP -nologo -Zc:wchar_t- -W3 -
Gy -Fdgenerated.pdb  -DDEBUG -D_DEBUG -DDEBUG_blassey -DTRACING -Zi -O1   -DCROS
S_COMPILE=1 -D_CRT_SECURE_NO_WARNINGS=1 -D_CRT_NONSTDC_NO_WARNINGS=1 -DWINVER=0x
500 -D_WIN32_WINNT=0x500 -D_WIN32_IE=0x0500 -DX_DISPLAY_MISSING=1 -DMOZILLA_VERS
ION=\"1.9.2a1pre\" -DMOZILLA_VERSION_U=1.9.2a1pre -DHAVE_SNPRINTF=1 -D_WINDOWS=1
 -D_WIN32=1 -DWIN32=1 -DXP_WIN=1 -DXP_WIN32=1 -DHW_THREADS=1 -DSTDC_HEADERS=1 -D
WIN32_LEAN_AND_MEAN=1 -DD_INO=d_ino -DMOZ_EMBEDDING_LEVEL_DEFAULT=1 -DMOZ_EMBEDD
ING_LEVEL_BASIC=1 -DMOZ_EMBEDDING_LEVEL_MINIMAL=1 -DMOZ_XULRUNNER=1 -DMOZ_BUILD_
APP=xulrunner -DMOZ_DEFAULT_TOOLKIT=\"cairo-windows\" -DMOZ_DISTRIBUTION_ID=\"or
g.mozilla\" -DIBMBIDI=1 -DMOZ_VIEW_SOURCE=1 -DMOZ_JSLOADER=1 -DMOZ_NO_XPCOM_OBSO
LETE=1 -DMOZ_OGG=1 -DMOZ_WAVE=1 -DMOZ_SYDNEYAUDIO=1 -DMOZ_MEDIA=1 -DMOZ_XTF=1 -D
MOZ_CRASHREPORTER_ENABLE_PERCENT=100 -DMOZ_MATHML=1 -DMOZ_ENABLE_CANVAS=1 -DMOZ_
SVG=1 -DMOZ_TREE_FREETYPE=1 -DHAVE_FT_BITMAP_SIZE_Y_PPEM=1 -DHAVE_FT_GLYPHSLOT_E
MBOLDEN=1 -DHAVE_FT_LOAD_SFNT_TABLE=1 -DMOZ_UPDATE_CHANNEL=default -DMOZ_DISABLE
_VISTA_SDK_REQUIREMENTS=1 -DMOZ_DISABLE_PARENTAL_CONTROLS=1 -DMOZ_PLACES=1 -DMOZ
_FEEDS=1 -DMOZ_STORAGE=1 -DMOZ_GFX_OPTIMIZE_MOBILE=1 -DMOZ_LOGGING=1 -DMOZ_USER_
DIR=\"Mozilla\" -DMOZ_ENABLE_LIBXUL=1 -DMOZ_TREE_CAIRO=1 -DHAVE_UINT64_T=1 -DMOZ
_XUL=1 -DMOZ_PROFILELOCKING=1 -DMOZ_RDF=1 -DMOZ_MORKREADER=1 -DMOZ_DLL_SUFFIX=\"
.dll\" -DMOZ_REFLOW_PERF=1 -DMOZ_REFLOW_PERF_DSP=1  -D_MOZILLA_CONFIG_H_ -DMOZIL
LA_CLIENT /c/hg/mozilla-central/netwerk/base/src/nsInputStreamChannel.cpp
nsInputStreamChannel.cpp
nsInputStreamPump.cpp
Building deps for /c/hg/mozilla-central/netwerk/base/src/nsInputStreamPump.cpp
      0 [main] sed 3052 open_stackdumpfile: Dumping stack trace to sed.exe.stack
dump
make[7]: *** [nsInputStreamPump.obj] Error 139
make[7]: Leaving directory `/c/hg/mozilla-central/objdir-wm6-dbg-cmd-cl/xulrunne
r/netwerk/base/src'
make[6]: *** [libs] Error 2
make[6]: Leaving directory `/c/hg/mozilla-central/objdir-wm6-dbg-cmd-cl/xulrunne
r/netwerk/base'
make[5]: *** [libs] Error 2
make[5]: Leaving directory `/c/hg/mozilla-central/objdir-wm6-dbg-cmd-cl/xulrunne
r/netwerk'
make[4]: *** [libs_tier_necko] Error 2
make[4]: Leaving directory `/c/hg/mozilla-central/objdir-wm6-dbg-cmd-cl/xulrunne
r'
make[3]: *** [tier_necko] Error 2
make[3]: Leaving directory `/c/hg/mozilla-central/objdir-wm6-dbg-cmd-cl/xulrunne
r'
make[2]: *** [default] Error 2
make[2]: Leaving directory `/c/hg/mozilla-central/objdir-wm6-dbg-cmd-cl/xulrunne
r'
make[1]: *** [build] Error 2
make[1]: Leaving directory `/c/hg/mozilla-central'
make: *** [build] Error 2

Comment 6

[Brad Lassey [:blassey] (use needinfo?)]

Attachment: stack dump from sed crash

this may be entirely unrelated....

Comment 7

[Brad Lassey [:blassey] (use needinfo?)]

Attachment: mozconfig used to build

this has a lot of stuff specific to my machine such as path short names.

Comment 8

[Brad Lassey [:blassey] (use needinfo?)]

Attachment: batch file used to set up build env

Comment 9

[(not currently active) Ted Mielczarek]

Do you reliably crash there? I've seen crashes in various msys utils before, I think it's just msys suckage.

Comment 10

[Brad Lassey [:blassey] (use needinfo?)]

Comment on attachment 360472 [details]
stack dump from sed crash

yup, just msys suckage.  I kicked off a new build before bed and it got all the way to sqlite where it died because RC wasn't set up right.

Comment 11

[Brian Crowder]

woops, wrong bug, sorry.

Comment 12

[Hiroyuki Ikezoe (:hiro)]

Add dependency based on comment #5.

Comment 13

[(not currently active) Ted Mielczarek]

That's unrelated, see blassey's followup comments.

Comment 14

[Brad Lassey [:blassey] (use needinfo?)]

Attachment: WIP

this successfully builds everything with pymake.  I've got a build with regular make going now.

the NSS bits are a hack, it'll need work.

Comment 15

[Brian Crowder]

Not doin' WinMo NoMo.