421095 - Hook up dehydra to a mingw build

Status: RESOLVED WONTFIX

(Developer Infrastructure :: Source Code Analysis, defect)

Description

[Benjamin Smedberg]

Right now there's a fair bit of Windows-specific code that we're not analyzing because we don't have dehydra-gcc on Windows/mingw. This is a tracking bug to cover whatever tasks are necessary to statically analyze our Windows codebase. I think that the quick path to success is to perform the actual compilation on Linux using a mingw cross-compile toolchain.

cls, do you have instructions for setting up a cross-compile mingw toolchain for Mozilla? And, does w32api have all the functions we need for the trunk codebase?

Comment 1

[cls]

I only have the generic cross-compile instructions that I published a long time ago: http://www.mozilla.org/build/cross-compiling.html .  My mingw cross-rpms haven't been updated in a couple of years but iirc, the build process wasn't that complex.  At least one person in the mingw community has published an all-in-one script to build the cross toolchain from scratch for you.

A couple of the open mingw-related bugs appear to point to w32api header discrepancies but I haven't dug to get any details.

Comment 2

[Jacek Caban]

I work on Wine (www.winehq.org) and we use our own Gecko package to provide MSHTML functionality. We build that package on Linux. The package it plain Gecko with a few Wine-specific hacks and a few paches to fix MinGW compilation. The instruction may be found on our wiki:
http://wiki.winehq.org/BuildingWineGecko
They are based on resources found on mozilla.org. It applies to package based on Firefox 3.1b2 source. Current Thunk needs much more patches to compile. I was able to fix compilation and I'm currently filling bug reports with patches.

I'm very interested in helping to setup such testing environment as it looks like MinGW build regresses very often.

Comment 3

[(dormant account)]

(In reply to comment #2)

> I'm very interested in helping to setup such testing environment as it looks
> like MinGW build regresses very often.

yes that'd be very good, if you could set it up with gcc 4.3(which would require fixing some compiler bugs).

Comment 4

[Jacek Caban]

GCC bug is fixed in GCC 4.4:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9381
Unfortunately it's not fixed fixed in 4.3 branch, but AFAIK dehydra is ported to 4.5, so we could target it on GCC 4.5.

Also bugs that I've marked as dependency of this bug together with a patch fixing compilation from bug 533035 (it is in TM tree) are all that's needed for successful mingw compilation (including tests) with GCC 4.4 from SVN. All these bugs are waiting for review or landing (I don't have permission to land them myself).

There are also some fixes of mingw importlibs and includes needed. I get them to official mingw myself (I'm not allowed due to disagreement on the way Wine and mingw create headers), but I believe it may be fixed locally the way I do.

I will clean up my compilation environment and write a HOWTO soon.

Comment 5

[Jacek Caban]

Here is an instruction how to setup compile environment and build source with mingw. It's based on how I build Gecko for Wine (that was based on old instruction found on mozilla.org):

----------------------
* Install binutils-2.20

$ wget http://ftp.gnu.org/gnu/binutils/binutils-2.20.tar.bz2
$ tar -jxf binutils-2.12.tar.bz2
$ cd binutils-2.20
$ ./configure --prefix=/usr/local --target=i486-mingw32
$ make
$ make install

* Install mingw-runtime-3.15.2

$ wget http://downloads.sourceforge.net/project/mingw/MinGW%20Runtime/mingwrt-3.15.2/mingwrt-3.15.2-mingw32-dev.tar.gz
$ mkdir mingw-runtime
$ cd mingw-runtime
$ tar -zxf ../mingwrt-3.15.2-mingw32-dev.tar.gz
$ cd ..
$ cp -fr mingw-runtime/* /usr/local/i686-mingw32

* Install w32api-3.14

$ wget http://downloads.sourceforge.net/project/mingw/MinGW%20API%20for%20MS-Windows/w32api-3.14/w32api-3.14-mingw32-dev.tar.gz
$ mkdir w32api
$ cd w32api
$ tar -zxf ../w32api-3.14-mingw32-dev.tar.gz
$ cd ..
$ cp -fr w32api/* /usr/local/i686-mingw32

* Extract winsup 

$ wget http://cygwin.com/snapshots/winsup-src-20091222.tar.bz2
$ tar -jxf winsup-src-20091222.tar.bz2

* Install gcc-4.4 SVN

NOTE: GCC had a bug (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9381) that was recently fixed.
It's not yet available in any release, so we have to use SVN version from 4.4 branch.
DO NOT USE ANY OTHER VERSION. Getting source is out of scope of this instruction
(hint: there is a Git mirror git://git.infradead.org/gcc.git). Further instructions assume
that you have the source in gcc-git directory.

$ cp -fr cygwin-snapshot-20091222-1/winsup gcc-git
$ mkdir gcc-mingw
$ cd gcc-mingw
$ ../gcc-git/configure --prefix=/usr/local --target=i686-mingw32 --with-gnu-ld --with-gnu-as --enable-__cxa_atexit --enable-languages=c,c++
$ make
$ make install

* Install wintools 

$ wget http://ftp.mozilla.org/pub/mozilla.org/mozilla/source/wintools.zip
$ unzip wintools.zip
$ mkdir /usr/local/src/moztools
$ cp -fr buildtools/windows/include /usr/local/src/moztools
$ cp -fr buildtools/windows/lib /usr/local/src/moztools
$ cd /usr/local/src/moztools/include
$ dos2unix *.h
$ cd libIDL
$ dos2unix *.h

* Replace importlibs and includes 
Headers and lib replacements are modified MinGW versions or based on Wine source.
You can get them from wine-gecko Git repository:
http://wine.git.sourceforge.net/git/gitweb.cgi?p=wine/wine-gecko;a=tree;f=wine/include;h=8a51b23f5ec349c4feecd4a7417a907aa3fda4b6;hb=HEAD
http://wine.git.sourceforge.net/git/gitweb.cgi?p=wine/wine-gecko;a=tree;f=wine/lib;h=89e221c1f0f608ee37e101b04bea60f5bf92098c;hb=HEAD
Copy them to /usr/local/i686-mingw32 directory (lib and include subdirectories).

* Build wine_gecko 
Use following mozconfig:
http://wine.git.sourceforge.net/git/gitweb.cgi?p=wine/wine-gecko;a=blob;f=wine/mozconfig-browser;h=9460e335e76e7b56f4a0d468618cf044a21cf4f5;hb=HEAD
There is nothing unusual about building it once you have the compile environment, except that there is a bug in build dependencies. If libgkdebug.a is missing during building xul.dll, build it manually:
$ make -C ../mozilla-build/extensions/layout-debug/
and continue regular building.

------------------------------------

I'd appreciate a comment if it's going in the right direction for dehydra usage.

Comment 6

[Joshua Cranmer [:jcranmer]]

(In reply to comment #5)
> * Install gcc-4.4 SVN

I believe Dehydra works for either gcc 4.3 or gcc 4.5 but not gcc 4.4, which would imply that the instructions would require gcc 4.5 instead of gcc 4.4...

Comment 7

[Jacek Caban]

Attachment: Patches required to compile with mingw

GCC 4.5 is in my plans, but it didn't work last time I tried. I'm attaching a diff of my tree against m-c in case it would be useful for someone (all it contains may be found in blocking bugs).

Comment 8

[Jacek Caban]

Attachment: updated patch

Comment 9

[Jacek Caban]

Attachment: updated patch

I'm attaching an updated diff of my tree. There are only two patches waiting for review, one for check-in and one for tracemonkey merge.

Unfortunately a new ugly hack is required to setup the environment:
$ mv /usr/local/lib/gcc/i686-mingw32/4.4.3/include/float.h /usr/local/lib/gcc/i686-mingw32/4.4.3/include/_float.h

There is also a new set of header fixes:
http://wine.git.sourceforge.net/git/gitweb.cgi?p=wine/wine-gecko;a=tree;f=wine/include;hb=refs/heads/master

Comment 10

[Jacek Caban]

GCC 4.4.3 is now released and it may be used to compile Mozilla. Only one patch needed to compile with mingw is not yet in the tree (from bug 505731).

GCC 4.5 causes some troubles. One bug in GCC is fixed now:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43031
but there is another that needs to be fixed:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43093

Comment 11

[Jacek Caban]

I haven't updated this bug for a while, so let me give an update.

I've landed the last patch required for mingw build this week (it's the first time review+landing process could catch new problems introduces, I have the tree in compilable state about all the time, so if anyone is interested, ping me).

GCC bugs are fixed and GCC 4.5 currently works fine (I use 4.5.2, but all 4.5.* versions should work). I've also switched my builds to use mingw-w64 instead of plain mingw. Short clarification: name may be misleading. It's a fork of mingw that supports both 32-bit builds (like plain mingw) and 64-bit builds (unlike plain mingw). It's even possible to compile 64-bit build, but it requires some not yet landed patches. mingw-w64 project has much better development model and currently is much better than plain mingw, so it will probably take place of mingw sooner or later.

Given all that, here is an instruction for 32-bit compilation. Note that it's much simpler than it used to be and no hacks are needed:

* Getting sources

First get required sources and extract them in a build directory. Note that
mingw-w64 SVN version is required.

$ wget http://ftp.gnu.org/gnu/binutils/binutils-2.21.tar.bz2
$ tar -jxf binutils-2.21.tar.bz2
$ wget http://gcc.fyxm.net/releases/gcc-4.5.2/gcc-4.5.2.tar.bz2
$ tar -jxf gcc-4.5.2.tar.bz2
svn co https://mingw-w64.svn.sourceforge.net/svnroot/mingw-w64/trunk mingw-w64-svn

* binutils

$ mkdir binutils-2.21-mingw32 && cd binutils-2.21-mingw32
$ ../binutils-2.21/configure --prefix=/usr/local/ --target=i686-w64-mingw32
$ make
# make install

* mingw-w64 headers

$ mkdir mingw-w64-headers32 && cd mingw-w64-headers32
$ ../mingw-w64-svn/mingw-w64-headers/configure --host=i686-w64-mingw32 --prefix=/usr/local/ --enable-sdk=all --enable-secure-api
# make install

* GCC compiler

$ mkdir gcc-4.5.2-mingw32 && cd gcc-4.5.2-mingw32
$ ../gcc-4.5.2/configure --prefix=/usr/local/ --target=i686-w64-mingw32 --with-gnu-ld --with-gnu-as --enable-__cxa_atexit --enable-languages=c,c++ --disable-multilib
$ make all-gcc
# make install-gcc

* mingw-w64 crt

$ mkdir mingw-w64-lib32 && cd mingw-w64-lib32
$ ../mingw-w64-svn/mingw-w64-crt/configure --host=i686-w64-mingw32 --prefix=/usr/local/
$ make
# make install

* Finnishing GCC

$ cd gcc-4.5.2-mingw32
$ make
# make install
# echo "#include_next <float.h>" >> /usr/local/lib/gcc/i686-w64-mingw32/4.5.2/include/float.h

(the last echo trick is not needed in 4.6 trunk GCC version anymore).

I will attach mozconfig that I use for compilation. There are many things disabled. That's on my TODO list to fix them, but most of windows-specific code is compiled and the built browser works.

Comment 12

[Jacek Caban]

Attachment: mozconfig

LDFLAGS are set purely to make testing easier and not require crt DLLs to be available.

Comment 13

[Joshua Cranmer [:jcranmer]]

Dehydra and treehydra are no longer maintained by Mozilla.