173066 - Hook modules-dep into --enable-standalone-modules

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, defect)

Description

[hacker formerly known as [e-mail]]

I think all that is really required at this point is to add a
--modules-file-only option to bootstrap.pl.  Then we can call bootstrap.pl with
a list of all of the known modules and just check in the generated modules.mk file.

Comment 1

[hacker formerly known as [e-mail]]

Attachment: v1.0

Yay! It works (mostly).  ./mozilla/tools/module-deps/bootstrap.pl
--module-file-only will generate modules.mk & allmakefiles.sh files which
contain enough info to let you build any of the 147 "modules" using
--enable-standalone-modules=mod1,mod2,etc.    Not all of the module definitions
are complete though (ie, xpconnect_test requires more than just
js/src/xpconnect/tests to build).  And the generated files are huge! 

Pulling using 'make -f client.mk pull_all  BUILD_MODULES=xpcom' needs some work
 still.  Right now I'm assuming that the directories returned by
module-graph.pl should not be pulled recursively.  However, in some cases (at
least dbm & xpcom), they do need to be pulled recursively.  

Changes:
* Removed unnecessary ifdefs from mozilla/Makefile.in to allow external modules
to be built in the same fashion for external builds.
* Removed unnecessary EXPORT_DIRS from rules.mk
* Added --module-file-only & --topsrcdir options to bootstrap.pl
* Moved some code of bootstrap.pl into subroutines so that it could be shared.
* Added modules.mk.footer to complement the modified modules.mk.stub
* Added global $topsrcdir & $toolsdir vars to bootstrap.pl because I got tired
of typing mozilla/...
* Modified module-graph.pl to spit out a list of all of the modules (not just
the toplevel ones) when --start-module = ALL .

Comment 2

[hacker formerly known as [e-mail]]

Attachment: v1.1

On the drive home, I decided to just recursively pull the dirs returned by
module-graph.pl.  This worked fine for pulling & building dbm & xpcom but cvs
complained alot when pulling content & content/xul/templates/src in the same
command.  I modified the unix.pl script to sort the dirs and do a regexp search
so that it won't pull a directory if one of its parent dirs have already been
pulled.   Atm, this requires that you set PERL when pulling using
BUILD_MODULES.

I also had to tweak the toplevel directory so that it would do the export-idl
hack whenever xpcom was in the build list.

Comment 3

[timeless]

note that js/ and layout/ have tests directories which are not part of SeaMonkey
but which you can get if you do some kinds of pulls. I'm not sure this matters,
and it might actually be useful for people doing standalone stuff to get this,
but I just felt it was worth noting.

Comment 4

[Alec Flett]

yeah, recursive directories are required right now, because we have poor lists
of directories for modules..we need to completely specify all the directories in
a module.

here's what someone really needs to do: write a script that walks the tree
calling "make echo-module" on each directory, and storing the combination of
{current dir, module} - then flip the map around we'll have a map from
module->{dir1, dir2, dir3} etc.

We should hold onto the script and run it periodically to keep this map up to
date. (this is why I really wanted the module->dir mapping to be in a seperate
data file)

Comment 5

[Chris McAfee]

Comment on attachment 102571 [details] [diff] [review]
v1.1

r=mcafee, talked to cls about this on the phone, let's do it.

Comment 6

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 102571 [details] [diff] [review]
v1.1

a=dbaron for trunk checkin.  (Spelling nit:  "pseudo", not "psuedo".)

Comment 7

[hacker formerly known as [e-mail]]

Patch has been checked in, along with the new modules.mk & allmakefiles.sh.  

The proposed module->dir mapping script should be in a separate bug since it
really only pertains to the meta-data.  I'm not sure if going that route is
going to help us though since we have the test directories which aren't part of
the binary build (as timeless pointed out) and the work in bug 167254 will cause
us to only have one Makefile for multiple source dirs.

Comment 8

[Axel Hecht]

activestate perl on WinME doesn't like the 
$(topsrcdir)/build/unix/uniq.pl
as that's starting with /cygdrive/...
It would need a cygpath -w plus sed, like in configure.in. I'm just not sure
how to hook this up in make.
cygwin perl was different with paths, IIRC.

Comment 9

[Axel Hecht]

... fixed, thanx cls.

Comment 10

[hacker formerly known as [e-mail]]

*** Bug 57282 has been marked as a duplicate of this bug. ***