177789 - (os2gcc3) [META] Make OS/2 Mozilla build with GCC 3.x

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, defect, P2)

Description

[jhp (no longer active)]

This is a Meta bug for work required to build the OS/2 version of Mozilla using
GCC 3.0.3 for EMX/OS2.  Most of the preliminary work will take place on this
bug, but eventually, the patches will have to split off into other bugs for
NSPR, LDAP, security, etc.

Comment 1

[jhp (no longer active)]

Attachment: preliminary patch

With this patch, the build finishes, but does not come up.  There are severl
TODO items, which I will post shortly.	Also, I will soon post a setup file
detailing where to get the tools (I am using newer and better tools than the
ones listed at www.mozilla.org/ports/os2/setup.html).

Comment 2

[jhp (no longer active)]

Attachment: GCC 3.0.3 Setup Guid

Setup guide with links to the tools.

Comment 3

[jhp (no longer active)]

Attachment: setmozenv.cmd

CMD file to set up build environment.  Run it like this: setmozenv gcc

Comment 4

[jhp (no longer active)]

TODO List (not in any order):
1) Write atomic ops in os2misc.c
2) Write ramsems in os2cv.c
3) Do we need to add "-ansi" to CFLAGS in nsprpub, like Linux does?
4) In nsDeviceContextSpecOS2.cpp, how do we use GRE functions (see diff)?  It is
not implemented in the EMX libs.
5) Several files give "emxomf warning: Cannot convert incomplete enum type". 
What does this mean, and how can we fix it?
6) Is it possible to depend more on OS/2 toolkit, rather than accessing toolkit
functions through emx?
7) Need to write correct xptcall code (xpcom\reflect\xptcall\src\md\os2).  Henry
suggested taking Linux files (..\unix\*_gcc_x86_unix.* files), but I get an ASM
error.  Still investigating.

Comment 5

[Julien Pierre]

Javier,

Are you able to debug the resulting binary with IDEBUG / JITDBG ?
This was the case with binaries built from EMX with gcc 2.9x .

Comment 6

[jhp (no longer active)]

Yes, I am able to debug using IDEBUG/JITBDG.

Also, please note that I did not build mailnews or tests yet (disabled in
.mozconfig), so those may break when you build.

Comment 7

[jhp (no longer active)]

Weak symbols:  As explained in the documentation, the linker creates a
'weaksyms.omf' file during it's conversion from a.out format to OMF format.  For
the Mozilla project, we need to specify one 'weaksyms.omf' file for the whole
build.  The environment variable GCC_WEAKSYMS tells the emxomf program where to
look for the file.  I usually do something like 'SET
GCC_WEAKSYMS=d:/builds/gcc/mozilla/obj/weaksyms.omf' (assuming an OBJDIR build).

However, I would like to be able to make this build dependent.  In other words,
when I switch to another tree on my machine, I don't want to have to manually
set this environment variable every time.  Does anyone have any idea how I could
take care of this in the Mozilla tree?  Perhaps by adding something to rules.mk?

This would not be an issue if the g++ command took a parameter to specify the
location of weaksyms.omf (i.e. "g++ --weaksyms-file=d:/foo/bar/weaksyms.omf"). 
I sent an email to two of the people associated with the development of GCC3 on
OS/2, but have yet to hear a response.  I would appreciate any help with this issue.

Comment 8

[jhp (no longer active)]

Attachment: GCC 3.0.3 Setup Guide

Updated setup guide with build instructions and GCC_WEAKSYMS environment
variable.

Comment 9

[jhp (no longer active)]

In trying to build tests, Henry suggested that I should not be defining
C_INCLUDE_PATH and CPLUS_INCLUDE_PATH.  After doing that and restarting the
build, I get a compile error in nsprpub\pr\src\io\prscanf.c:

 prscanf.c:298: first argument to `va_arg' not of type `va_list'

It compiled fine previously, but I was probably pulling in the incorrect
headers.  So I just preprocessed the file, to see what was getting included, and
the file stdio.h in defining 'va_list' as a 'char*', which is probably causing
these problems.
I have not fixed this yet, so I am putting this up to see if anyone can work out
the header issues.  

Comment 10

[sobotka]

IIRC C_INCLUDE_PATH is needed in some circumstances.

The workaround for the va_arg problem is to neutralize (rename/move/whatever)
emx/lib/gcc-lib/i386-pc-os2_emx/3.0.3/include/stdarg.h.

Comment 12

[hacker formerly known as [e-mail]]

Comment on attachment 104793 [details] [diff] [review]
preliminary patch

This patch should be split into separate parts for NSPR & LDAP.

Comment 13

[jhp (no longer active)]

Chris:  Yes, I know that it needs to be split.  At the moment, though, the stuff
isn't ready for checkin.  There is still much that can change, so I am just
leaving it as a large patch here for anyone who would like to help.  When the
patch becomes a little more stable and is nearing the time to checkin, then I
will split it in three.  Thanks.

Comment 14

[jhp (no longer active)]

Attachment: setmozenv.cmd

Fixes CPLUS_INCLUDE_PATH and some other env variables.

Comment 15

[jhp (no longer active)]

Attachment: setmozenv.cmd

Sorry, previous patch had a few typos.

Comment 16

[jhp (no longer active)]

Attachment: patch v0.2 (based on trunk)

Trunk patch: Now Mozilla builds fully (but does not load) with the following
.mozconfig:
  ac_add_options --disable-mailnews
  ac_add_options --disable-installer

TODO items:
-----------
1) Write the xptcall code.  Supposedly, we should be able to take some of the
linux GCC 3.x code;  I'm worried though that it was all for GCC 3.2.  Anyone
know if code was written for GCC 3.0?
2) EMX/GCC don't define strcmpi (see many references in patch).  Need to find a
central file where I can add this so all files pick it up.  May just need to
put it in configure.in to include in mozilla-config.h file.
3) Any other "TODO" items referenced in patch or in comment #4.
4) Run tests in dist/bin and make sure everything works.  However, some of
these tests require that the xptcall code be written first.

Comment 17

[hacker formerly known as [e-mail]]

The gcc3 xptcall code was originally written for gcc3.0.  Afaik, it hasn't
changed with gcc3.2 and should still be compatible.

Comment 18

[sobotka]

Last time I built with 3.0.2 was 0.9.8 and the linux i386 code worked
out-of-the-box  on OS/2. Oddly enough, it doesn't with earlier versions of gcc
on OS/2.

Comment 19

[jhp (no longer active)]

Split up the patch and opened 3 new bugs:
  188246: Build NSPR with 3.0.3
  188247: Build LDAP with 3.0.3
  188249: Build rest of browser with 3.0.3
So this bug should be considered the 'parent' bug, and all patches should be
done in the listed bugs (or new ones that are opened).  The only thing that can
stay here are the 'Setup Guide' and 'setmozenv.cmd' script.

Comment 20

[jhp (no longer active)]

Attachment: setmozenv.cmd

Comment 21

[jhp (no longer active)]

Attachment: ilinkwrap.exe (wrapper for ilink.exe; this is a zip file)

Comment 22

[jhp (no longer active)]

Attachment: GCC 3.x Setup Guide

Comment 23

[Andy Willis (abwillis)]

GNUPatch needs to be added as required file to install patches.

Comment 24

[Andy Willis (abwillis)]

Moztools gets defined twice in setmozenv.cmd
first as:
'SET MOZTOOLS=%ROOT%\MOZTOOLS'
where it is defined for everything
which is good 
and then as:
'SET MOZTOOLS=D:\MOZTOOLS'
under VACPP
which isn't so good (mine is on e: for instance).
Because of where it is in the scheme of things it doesn't hurt but it is unneeded.
It should be removed path and beginlibpath under this section as well.
Trying to think of a better weaksysms strategy but I don't know that it is any
harder to make changes to the hard coding that is there now than to set a
variable for it the more I look at it.

Comment 25

[jhp (no longer active)]

Attachment: setmozenv.cmd

Updated INCLUDE paths

Comment 26

[jhp (no longer active)]

Attachment: GCC 3.x Setup Guide

Corrected link to autoconf

Comment 27

[jhp (no longer active)]

The GCC setup guide is now at http://www.mozilla.org/ports/os2/gccsetup.html.  I
just recently updated it with the location for nsinstall.exe.

Comment 28

[sobotka]

Javier, the link for the Perl core zip should be pub/os2/dev/perl rather than
pub/new.

Comment 29

[Andy Willis (abwillis)]

I had to make a change to objs.mk in nsprpub\pr\src\md\os2
I had to add $(NULL) as such:
OBJS += $(addprefix md/os2/$(OBJDIR)/,$(CSRCS:.c=.$(OBJ_SUFFIX)))  \
	$(addprefix md/os2/$(OBJDIR)/,$(ASFILES:.$(ASM_SUFFIX)=.$(OBJ_SUFFIX))) \
	$(NULL) <=====
Not sure why as I take it no one else has to or it would be there.  Without it
it is not stripping the .s and changing it to a .o as it should.  The only thing
I can think of is that I am unable to use the make that is listed on the setup
page as it just dies with a diagnostic error 5 so I had to use the next release
of it.  Don't know if that would have any affect or not though.  This is the one
I am having to use:
http://hobbes.nmsu.edu/pub/os2/dev/util/make-3_79_2a1-bin.zip
And I am having to delete configure from the nsprpub directory after a cvs pull
and let it recreate or the build fails.
Also:
To get past:
TestCOMPtr
I had to add %EMX%\include\c++\3.2.1;
to the beginning of the include line as such:
'SET
INCLUDE=%EMX%\include\c++\3.2.1;%EMX%\include\g++-v3;%EMX2%/lib/gcc-lib/i386-pc-os2-emx/3.2.1/include;%EMX%\include;%INCLUDE%'
it kept failing with the stl_alloc.h in %EMX%\include\g++-v3\bits.  This change
forced it to use the one in %EMX%\include\c++\3.2.1\bits which worked.
The ones in g++ are older but there are more files there than in c++ so I think
both may be required.
Don't know of anything else yet as the next place I am dieing I haven't figured
out yet.

Comment 30

[jhp (no longer active)]

Andy:  Now that I look into this, it appears that this may be left over from the
GCC 3.0.3 build.  From a clean GCC 3.2.1 install, I do not have a "g++-v3" path,
only the c++.  Can you confirm this?  Did you install 3.2.1 on top of 3.0.3?

I will try to build with the CPLUS_INCLUDE_PATH set up correctly.

Comment 31

[Andy Willis (abwillis)]

I installed over 3.0.3 and just unzipped into a fresh directory and it does not
have g++.  CPLUS_INCLUDE_PATH had the c++ (also had g++) but include had just
g++ and it seemed to take precedence.  
Andy

Comment 32

[jhp (no longer active)]

Andy:  I removed the 'set INCLUDE=...' line from my setmozenv.cmd and was able
to build without any problems.

Comment 33

[Andy Willis (abwillis)]

I am at a loss (again).  I am now dieing in NSS-

make.exe[5]: Entering directory `E:/CVS/WORK/MOZILLA/security/nss/lib/cryptohi'
make.exe[5]: $SHELL changed (was `/bin/sh', now `bash.exe')
nsinstall              -R -m 444 cryptohi.h cryptoht.h hasht.h key.h keyhi.h
keyt.h keythi.h sechash
.h E:/CVS/WORK/MOZILLA/obj/dist/public/nss
There are no private exports.
make.exe[5]: Leaving directory `E:/CVS/WORK/MOZILLA/security/nss/lib/cryptohi'
Skipping non-directory crypto...
cd nss; make.exe -j1 export
make.exe[5]: Entering directory `E:/CVS/WORK/MOZILLA/security/nss/lib/nss'
make.exe[5]: $SHELL changed (was `/bin/sh', now `bash.exe')
../../../coreconf/rules.mk:327: *** multiple target patterns.  Stop.

ifeq ($(OS_TARGET),OS2)                                                        
                                                                               
                   
$(IMPORT_LIBRARY): $(MAPFILE)                                                  
                                                                               
                   
	rm -f $@                                                                      
                                                                               
                   
	$(IMPLIB) $@ $(MAPFILE)                                                       
                                                                               
                   
	$(RANLIB) $@                                                                  
                                                                               
                   
endif                                                                          
                                                                               
                   
                                                                               
                                                                               
                   
ifdef SHARED_LIBRARY_LIBS                                                      
                                                                               
                   
ifdef BUILD_TREE                                                               
                                                                               
                   
SUB_SHLOBJS = $(foreach dir,$(SHARED_LIBRARY_DIRS),$(shell $(MAKE) -C $(dir)
--no-print-directory get_objs))                                                
                      
else                                                                           
                                                                               
                   
SUB_SHLOBJS = $(foreach dir,$(SHARED_LIBRARY_DIRS),$(addprefix $(dir)/,$(shell
$(MAKE) -C $(dir) --no-print-directory get_objs)))                             
                    
endif                                                                          
                                                                               
                   
endif                                                                          
                                                                               
                   
                                                                               
                                                                               
                   
$(SHARED_LIBRARY): $(OBJS) $(RES) $(MAPFILE) $(SUB_SHLOBJS)  <===This is  line
327                                                                            
            
	@$(MAKE_OBJDIR)                                                               
                                                                               
                   
	rm -f $@                                                                      
                                                                               
                   

Don't know what I need to change here.
Andy

Comment 34

[Mike Kaply [:mkaply]]

Which GCC did you grab? It sounded like you grabbed the latest one from hobbes
which has a bug.

Please grab the one referenced in:

http://www.mozilla.org/ports/os2/gccsetup.html

Comment 35

[jhp (no longer active)]

Not GCC.  You need the latest make (make-3_79_2a1-r2-bin.zip).  It's on the page
that Kaply referenced.

Comment 36

[Andy Willis (abwillis)]

Maybe I will reload both.  I have both currently, I think I still have them all
downloaded so it will just take a little time to try it.

Comment 37

[Andy Willis (abwillis)]

When going over the instructions when I was going to reinstall gcc I realized
that there had to be another make on my system to have set it up originally.  I
removed it and finally got a complete build :)
Unfortunately launching mozilla.exe gives:

*** Registering nsSoftwareUpdate components (all right -- a generic module!)
nsNativeComponentLoader: autoregistering succeeded

Process terminated by SIGSEGV
SYS1808:
The process has stopped.  The software diagnostic
code (exception code) is  0005.

Comment 38

[jhp (no longer active)]

This is strange, but you need to apply the patch from bug 190538 in order to get
past this crash.  We still haven't found out why.  You will need to rebuild
everything though.

Comment 39

[Andy Willis (abwillis)]

I just noticed that the intl.dll
# gettext-0_11_5-r2-bin.zip
    * intl.dll
is not the one that is being used.  Moztools gets put at the end of the
beginlibpath statement so it uses the one in emx\dll.
'SET
BEGINLIBPATH=%PERLLOC%\lib;%GLIB%\bin;%LIBIDL%\bin;%EMX%\dll;%MOZTOOLS%;%BEGINLIBPATH%'
Andy

Comment 40

[Andy Willis (abwillis)]

Looking through the gccsetup page I have a question on it.  The instructions I
had for setting up gcc for Odin included a step of running omflibs.cmd.  They
ran it after applying the fixes but before installing the gcc 3.x.x.  I don't
really know what it is doing so I can't say if it is really needed.
Andy

Comment 41

[Andy Willis (abwillis)]

I am not sure why this suddenly popped up but if I change the uint to uint16
(and I am sure uint8 or uint32 would work) the errors will go away but I don't
know why I suddenly am getting the errors nor the proper fix:
In file included from E:/CVS/WORK/MOZILLA/dbm/src/db.c:54:
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:331: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:332: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:333: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:334: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:335: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:347: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:347: warning: no semicolon at end of
struct or union
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:350: parse error before "psize"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:350: warning: type defaults to `int'
in declaration of `ps
ize'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:350: ISO C forbids data definition
with no type or storage
 class
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:356: parse error before '}' token
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:356: warning: type defaults to `int'
in declaration of `BT
REEINFO'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:356: ISO C forbids data definition
with no type or storage
 class
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:363: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:363: warning: no semicolon at end of
struct or union
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:364: warning: type defaults to `int'
in declaration of `ff
actor'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:364: ISO C forbids data definition
with no type or storage
 class
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:365: parse error before "nelem"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:365: warning: type defaults to `int'
in declaration of `ne
lem'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:365: ISO C forbids data definition
with no type or storage
 class
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:366: parse error before "cachesize"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:366: warning: type defaults to `int'
in declaration of `ca
chesize'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:366: ISO C forbids data definition
with no type or storage
 class
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:370: parse error before '}' token
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:370: warning: type defaults to `int'
in declaration of `HA
SHINFO'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:370: ISO C forbids data definition
with no type or storage
 class
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:378: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:378: warning: no semicolon at end of
struct or union
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:379: warning: type defaults to `int'
in declaration of `ps
ize'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:379: ISO C forbids data definition
with no type or storage
 class
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:384: parse error before '}' token
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:384: warning: type defaults to `int'
in declaration of `RE
CNOINFO'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:384: ISO C forbids data definition
with no type or storage
 class
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:450: warning: type defaults to `int'
in declaration of `BT
REEINFO'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:450: parse error before '*' token
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:451: warning: type defaults to `int'
in declaration of `HA
SHINFO'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:451: parse error before '*' token
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:452: warning: type defaults to `int'
in declaration of `RE
CNOINFO'
E:/CVS/WORK/MOZILLA/dbm/include/mcom_db.h:452: parse error before '*' token
E:/CVS/WORK/MOZILLA/dbm/src/db.c: In function `dbopen':
E:/CVS/WORK/MOZILLA/dbm/src/db.c:114: parse error before "HASHINFO"
E:/CVS/WORK/MOZILLA/dbm/src/db.c: In function `__dbpanic':
E:/CVS/WORK/MOZILLA/dbm/src/db.c:138: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/src/db.c:140: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/src/db.c:141: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/src/db.c:142: parse error before "uint"
E:/CVS/WORK/MOZILLA/dbm/src/db.c:143: parse error before "uint"
make.exe[4]: *** [db.o] Error 1

Andy

Comment 42

[sobotka]

omflibs.cmd converts aout .a libraries to OMF .lib ones; it's a wildcard wrapper
for emxomf. It's normally run when you install EMX and, after putting new .a
libs in emx/lib if you also want .lib versions. It's also run by the makefile
that gcc 3.x puts into emx/lib.

The uint error looks like a header with the required typedefs is missing or not
being found. Check the error output above the snippet you posted for a
header-not-found message; otherwise it's a missing #include, but that seems less
likely.

Comment 43

[Andy Willis (abwillis)]

The intl.dll being in emx was a mistake.  It was in my directory from when I
installed for Odin and the zip file was in my gcc directory when I wiped emx and
reinstalled everything it got installed too.  It might be a good idea to put
moztools in the front of path and libpath though.  I have myself run into
several problems of wrong versions of the tools being in the path/libpath prior
to moztools.
Andy

Comment 44

[jhp (no longer active)]

Andy, the build break in DBM is fixed now.  Also, the other NSPR patch went in,
so you should be able to build straight from CVS.  Once I confirm this, then I
will close out this bug.

Comment 45

[Andy Willis (abwillis)]

I died in building npnulos2.dll.  It couldn't find os2.h.  I did a set
include=%EMX2%/include/c++/3.2.1;%EMX2%/lib/gcc-lib/i386-pc-os2-emx/3.2.1/include;%EMX2%/include;
and it was able to build past it (currently building mailnews component).  Which
leads me to wondering what your include is set to as it is no longer set in
setmozenv.  I had it set to 'set include=' to clear it out as it was set to
E:\cvs\work\mozilla]set include
NCLUDE=e:\compilers\Ibmcxxo\INCLUDE;e:\Toolkit\som\include;e:\Toolkit\inc;e:\Toolkit\H\ARPA;e:\Tool
it\H\NET;e:\Toolkit\H\NETINET;e:\Toolkit\H\NETNB;e:\Toolkit\H\RPC;e:\Toolkit\SPEECH\H;e:\Toolkit\H\
L;e:\Toolkit\H;e:\Toolkit\H\LIBC;e:\Toolkit\H\MACHINE;e:\Toolkit\H\SYS;e:\Toolkit\H\STACK16;e:\Tool
it\H\PROTOCOL;.;E:\JAVA131\INCLUDE;
by the 'CALL %VACPP365%\bin\setenv'.  
It looks very promising that this build will complete and I will try again
tommorrow without doing a 'set include=' or adding the include.
Andy

Comment 46

[Andy Willis (abwillis)]

I am writing this from a successful build using GCC 3.2.1 YAY!  I will test
without changing the include tommorrow.
Andy

Comment 47

[Andy Willis (abwillis)]

It dawned on me this morning what must be happening.  I got the error during the
RC process.  RC.exe would be using the set include.  Because I had 'set
include=' it didn't have anywhere to search.   As long as the needed include
files are in the toolkit and not relying on the ones in vacpp then everything
should be fine as it is.
Andy

Comment 48

[hacker formerly known as [e-mail]]

-> javier

Comment 49

[Mike Kaply [:mkaply]]

We're done here.