281203 - xulrunner build fails on OS/2

Status: RESOLVED FIXED

(Toolkit :: Startup and Profile System, defect)

Description

[Andy Willis (abwillis)]

User-Agent:       Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.8b) Gecko/20050201
Build Identifier: Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.8b) Gecko/20050201

XULRunner build fails on OS/2 due to missing library.
Will upload patch to makefile.in

Reproducible: Always




The patch to makefile.in fixes the problem of the missing library but I now get
another error that I still have to find how to fix:
emximp -o xul.lib xul.def
emximp: Library too big -- increase page size
make.exe[1]: *** [xul.lib] Error 2
make.exe[1]: *** Deleting file `xul.lib'
make.exe[1]: Leaving directory `E:/cvs/work/mozilla/xulobj/toolkit/library'
make.exe: *** [all] Error 2

Comment 1

[Andy Willis (abwillis)]

Attachment: makefile.in adding -lpmwinx

Comment 2

[Andy Willis (abwillis)]

Attachment: include emximp page size increase

Comment 3

[Andy Willis (abwillis)]

The main question I have as to increasing the page size here is if it will
affect the rest of the build or just this section,

Comment 4

[Andy Willis (abwillis)]

Attachment: patch to find zlib libraries

I got unresolved ixternals in xpinstall.  I don't know why I am getting them as
it looks like the same code there that mozilla uses and it builds fine.  This
works around that but I don't know why I needed it.

Comment 5

[Andy Willis (abwillis)]

I think the answer is in autoconf.mk in lines 333 - 345.  

Comment 6

[Andy Willis (abwillis)]

Attachment: correct the ico that splash02.rc references

Splashos2.rc was apparently brought over directly from firefox and the firefox
ico was still being looked for.

Comment 7

[Andy Willis (abwillis)]

I had thought I had put the errors in here the errors in xpinstall.  The
unresolved externals were MOZ_Z_INFLATE, MOZ_Z_CRC32, MOZ_Z_INFLATEINIT,
MOZ_Z_INFLATE32.  
Also from the best that I can observe, it is safe to change the implib in
toolkit\library without affecting the rest of the build.
Bug 280137 will remove the dependency on -lpmwinx once it is checked in.

Comment 8

[Andy Willis (abwillis)]

Attachment: remove -lpmwinx

Removing the -lpmwinx and setting dependency on bug 280137 as it will be in
before this one and removes the pmwinx dependency.

Comment 9

[Mike Kaply [:mkaply]]

Can you post how to build xulrunner so I can try?

Or where the instructions are?

Comment 10

[Andy Willis (abwillis)]

http://wiki.mozilla.org/index.php/XUL:Xul_Runner
I am pulling from Trunk.
Basically it is about like Firefox in that it is just using the right .mozconfig
file.  
Here is what is on the above page:
$ cat $MOZCONFIG
export MOZILLA_OFFICIAL=1
mk_add_options MOZILLA_OFFICIAL=1

mk_add_options MOZ_CO_PROJECT=xulrunner
ac_add_options --enable-application=xulrunner

ac_add_options --enable-debug
ac_add_options --disable-optimize

I didn't add the last 2 options so I am going to add those and see what happens.  
The changes I have here got it built for me but the resulting executable pegs
the cpu at 100% and if I leave it there for very long I have to ctrl-alt-num-num
to get out.  I have built with both gcc 3.2.2 and 3.3.5.
I was really expecting this to work as much of this is now implemented in
Firefox as I understand it.

Comment 11

[Andy Willis (abwillis)]

This is the first I have tried to build since bug 280137 was checked in. 
Unfortunately this is also the first I have built with debug so I am not sure
which is the cause of this error yet.  Updating here so that I don't loose the
error, am going to rebuild with debug disabled to determine which is the cause.
weakld: xul.def - error: Parse error 65313 on line 10. (errorcode=17 stmt=117437
6)
weakld: error: Parse error 65313 on line 10. (errorcode=17 stmt=1173940)
E:\COMPILERS\GCC322\lib\dll0.obj({standard input}) : fatal error LNK1064: out of
 memory - heap exhausted
make.exe[1]: *** [xul.dll] Error 1
make.exe[1]: *** Deleting file `xul.dll'
make.exe[1]: Leaving directory `E:/cvs/work/mozilla/xulobj/toolkit/library'
make.exe: *** [all] Error 2

Comment 12

[Andy Willis (abwillis)]

I removed the enable debug and disable optimize and xul.dll built fine.

Comment 13

[Mike Kaply [:mkaply]]

(In reply to comment #12)
> I removed the enable debug and disable optimize and xul.dll built fine.

The parse error did not happen in that case?

Comment 14

[Andy Willis (abwillis)]

Correct:
with these lines I got the parse error:
ac_add_options --enable-debug
ac_add_options --disable-optimize
without those lines, no parse error.

Comment 15

[Mike Kaply [:mkaply]]

That's weird. We'll have to compare the generated xul.def in those two cases.

Comment 16

[Mike Kaply [:mkaply]]

we'll need to get the GCC folks to look at this I think.

The def files are the same at least until the failure.

I think the def file is too large for GCC to handle.

Comment 17

[Andy Willis (abwillis)]

Not sure if you contacted them but I put a post on the Innotek GCC forum.

Comment 18

[Andy Willis (abwillis)]

Posted this on the GCC forum as well but just building debug with 3.3.5 and it
doesn't get quite as far into the process:

make.exe[1]: Leaving directory `E:/cvs/work/mozilla/xulobj/docshell/build'
E:/MOZTOOLS/make.exe libs
make.exe[1]: Entering directory `E:/cvs/work/mozilla/xulobj/docshell/build'
rm -f docshell.lib
emxomfar r docshell.lib nsDocShellModule.o ../../dist/lib/basedocshell_s.lib ../
../dist/lib/uriloaderbase_s.lib ../../dist/lib/exthandler_s.lib ../../dist/lib/p
refetch_s.lib ../../dist/lib/shistory_s.lib
Creating library file `docshell.lib'
emxomfar: docshell.lib(../../dist/lib/exthandler_s.lib): Source library too big
make.exe[1]: *** [docshell.lib] Error 2
make.exe[1]: Leaving directory `E:/cvs/work/mozilla/xulobj/docshell/build'
make.exe: *** [all] Error 2

Comment 19

[knut st. osmundsen]

I'm a little bit short on time right now, but I'll get back to this soon.

Comment 20

[Andy Willis (abwillis)]

Additional workaround for building xulrunner due to ilink bug:
After building xulrunner (with ilink 5) I:
1.  change 3 lines in the xul.def (in toolkit\library):
  a.  ; segment bss32 (not quite right but it is like the 5 line)  I change the
; to rem
  b.  ; segment text32   change the ; to rem
  c.  ; segment data32  change the ; to rem
2.  run sed -e "s/;.*$/ NONAME/" xul.def  >xul.sav
3.  Change the rems back to ;  (otherwise the sed above would have changed the
lines)
4.  Then copy xul.sav to xul.def and set the read only attribute on it.
5.  Set the linker to use the 3.08 linker
  a.  change path and beginlibpath to point to linker from vac 3.08 
  b.  set EMXOMFLD_TYPE=VAC308
6.  Run make in toolkit\library. 

Long term fix will require:
1.  Get the linker fixed or
2.  Break xul.dll into 5 or 6 smaller pieces. or
3.  Somehow ifdef the makefile to run the sed script and switch linkers (might
have to require older linker for entire build for this to work but not sure).

Comment 21

[Mike Kaply [:mkaply]]

Do we understand what exactly the bug is in ilink?

I just can't imagine something working with the 308 linker and not ilink...

Comment 22

[knut st. osmundsen]

It's generating the wrong export table for the dll if the number of entries is
~14000 or ~50000(?) (I've only seen two examples yet). I don't know exactly with
how many it triggers. If I recall correctly in the first instance I examined the
export table was begin generated incorrectly start around export entry 12-13000.
Whether it was an extra byte or an incorrect by I really cannot recall right now.
Btw, we know it works fine with ~7000 exports (old debug xpcom.dll).

Comment 23

[Andy Willis (abwillis)]

One other piece for anyone trying to build... found out that a non-debug,
non-optimized build fails.  Right now a non-debug, optimized build is the only
one I am able to build.

Comment 24

[Andy Willis (abwillis)]

I don't know sed but got to playing around some and simplified the process some.
By changing the sed script to sed -e "s/; mag.*$/ NONAME/" xul.def >xul.sav
their is no further processing needed for the ; segment lines.

Comment 25

[Mike Kaply [:mkaply]]

> mkaply Rich found the xulrunner problem... nstoolkitprofile has the slashes the
> wrong way. Also I can't find the other changes I did to get firefox static to
> work so they must have been needed for something else and already got checked in

Interesting. I fixed a similar problem in Firefox when I was doing the rewrite:

http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=AVIARY_1_0_20040515_BRANCH&branchtype=match&dir=&file=&filetype=match&who=mkaply%25us.ibm.com&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2004%2F05%2F25&maxdate=2004%2F05%2F28&cvsroot=%2Fcvsroot

Check out those patches.

Comment 26

[Mike Kaply [:mkaply]]

Nope, those aren't it. I distinctly remember dealing with something like this...

Comment 27

[Mike Kaply [:mkaply]]

ok, I found the conversation

http://groups-beta.google.com/group/netscape.public.mozilla.os2/browse_thread/thread/5d3a503b504d9862/ce9a5ef707ec641a?q=getcwd#ce9a5ef707ec641a

Comment 28

[Andy Willis (abwillis)]

Attachment: all fixes but zlib wrapped up

Includes static build stuff

Comment 29

[Andy Willis (abwillis)]

Current status:
If ac_add_options --disable-libxul is added to the .mozconfig then it will build
without the workarounds as xul.dll is not built (debug builds are possible too).
 In order to build libxul then the workarounds are currently necessary and will
require one of the options in comment #20 or possibly _just_ exporting the
necessary exports may make it small enough.  If the exports are considerably
fewer then the emximp page size increase will not be necessary (not sure by how
much it would have to be decreased for that to be true).
At this point the only thing that doesn't work is the libxul builds without
workarounds (debug not at all) but these are due to tool limitations (or code if
we can reduce the exports programatically).

Comment 30

[Andy Willis (abwillis)]

Attachment: Another static patch.

Comment 31

[Andy Willis (abwillis)]

Attachment: updated static patch

Comment 32

[Andy Willis (abwillis)]

Attachment: shortname changes for simpletest.dll to simpltst

Comment 33

[Mike Kaply [:mkaply]]

All these patches are now checked in.

Is this bug done?

Comment 34

[Andy Willis (abwillis)]

Depends on if we are likely to get the fixes for the linker, and whether Knut
will have time to fix the problems with building debug libxul version.  As is we
can build the non-libxul but the writing on the wall suggests we are going to
want libxul if at all possible.  There are 2 other parts to the patches in a
couple of makefile.in's that aren't needed except for libxul builds.  As is the
only way to do that requires the VAC 3.0.8 linker and we can't do debug builds
even then.  I'll leave the call up to you.  If we close this out and are then
able to actually get the linker fixed then we can always open a new bug at that
time.

Comment 35

[Andy Willis (abwillis)]

Attachment: Patch for static sunbird

As this bug is still open and the other static build issues are in here I am
attaching the patch for static sunbird.  I hadn't tried to build it so didn't
notice the simple fact that it was one level off.

Comment 36

[knut st. osmundsen]

FYI. I've just ported the __declspec(dllexport) stuff from the cygwin/mingw gcc
port, so with a bit of hacking we can make use of these instead of exporting
every global symbol when building with gcc 3.3.5 and later. (Libc 0.6 is nearly
feature complete and will just need some more testing and bugfixing before I put
out a GA.)

Kind Regards,
 knut

Comment 37

[Peter Weilbacher]

Attachment: emxomfld error

I tried to build xulrunner today and encountered an error from emxomfld I show
in this attachment. Not sure what's causing this, this was a build with gcc
3.2.2. I tried reordering the parameters for the emxomfld call according to the
message and run that hy hand, but that didn't help. Does emxomfld have problems
with long parameter lines? This one has 2399 chars...

Btw, Mike, you said that all patches were checked in but the line with IMPLIB
from attachment 177397 [details] [diff] [review] does not show up in the CVS source (but it doesn't help
for my problem, either). Would it be possible to edit the attachments here in
bugzilla after checkin with a comment "(checked in to trunk)" or so? That might
make your and our life a bit easier...

Comment 38

[Peter Weilbacher]

Andy, did you try to build and run XULRunner recently? Did Knut's __declspec patch of bug 301064 help?

Comment 39

[Andy Willis (abwillis)]

Declspec does allow me to build xulrunner without disabling libxul.  The only issue with that is that it is a large dll so it uses a good bit of shared memory (not sure that it really uses more than seamonkey or firefox -or even xulrunner with libxul disabled- does with several smaller dlls).  I was building statically which isn't working now and when I asked on IRC was told that static builds weren't supported.

Comment 40

[Mike Kaply [:mkaply]]

xulrunner is building and declspec is in.