57282 - Building components selectively

Status: VERIFIED DUPLICATE of bug 173066

(SeaMonkey :: Build Config, defect, P2)

Description

[Henri Sivonen (:hsivonen)]

Currently the build system attempts pull the entire SeaMonkeyAll from CVS even if you 
only want to build some components. As a result, building selected components of 
Mozilla is not feasible in a disk-space-challanged environment.

It would help if the build system pulled only the sources required to build selected 
components.

Comment 1

[cls]

This can already be done to a point on the unix side.  

cvs -z3 co mozilla/client.mk
cd mozilla
make -f client.mk pull_all BUILD_MODULES=xpcom,js
./configure --enable-modules=xpcom,js

The list (build/unix/modules.mk) is far from complete, but it's a start.  

I'm not sure what the plans are for duplicating this functionality on windows or
mac.

Comment 2

[Simon Fraser [no longer active]]

To do selective checkouts and builds on Mac, you'll have to create new files for 
the build. You should clone and modify the following files

MozillaBuildFlags.txt
MozillaCheckoutList.txt
BuildMozillaDebug.pl
BuildMozilla.pl
PullMozilla.pl

and possibly

MozillaBuildList.pm

if you can't leverage the existing project groupings.

Comment 3

[cls]

At first glance, there appears to be no way to avoid building certain modules
standalone without losing functionality (via ifdefs) or building much more than
is needed for the standalone case (upto 10 additional modules depending upon
dependencies).  Case-in-point, xpcom.  To build xpcom standalone means that you
lose support for intl string handling in the nsCRT & nsLocalFile classes among
other things.  But to build the libraries required for intl string handling
means dealing with another slew of deps outside of xpcom (like prefs, js, necko:
and then deal with all of necko's deps).  Now, in xpcom's case, the extra module
deps are actually components so (in theory) they should be runtime optional.

So, assuming that the goal of a standalone module is to encompass the same
functionality as the non-standalone version, the ifdef solution needs to be
shown the door.  I'm not extremely familiar with other COM implementations but I
was under the impression that each module that depends upon a component needs to
Do The Right Thing(tm) if the component is not available.  So assuming this is
true, then all we really need to worry about is the external headers that
declare the component interfaces used by a module.

1) The first option is to modify the existing standalone build arch to contain a
list of dirs that are only traversed to export headers which should get rid of
the cascading dependency problems.

2) The second option is to create a toplevel interfaces/ directory that would
contain the actual public headers for the modules.  For starters, I'm thinking
about just moving the .idls to interfaces/.  This way, we can clearly outline
what the public interfaces of a particular module are and people who want to
build xpcom or some other intl depend module standalone do not have to pull the
entire intl source.

Option 1 is fairly straightforward.  The changes would not affect the normal
build process.  The only possibly side-effect would be if a module did something
non-standard during the export phase (which most do not).  And the downside is
the additional source directory requirements.

Option 2 is quite a bit more complicated and it does require changing the normal
build process.  It does allow us to clearly outline what the public interfaces
of a particular module are and people who want to build xpcom or some other intl
depend module standalone do not have to pull the entire intl source to use it.

Any thoughts?
 

Comment 4

[cls]

Attachment: Option 1 implemented for xpcom

Comment 5

[cls]

To test, you need to do this:

cvs -z3 co mozilla/client.mk
cd mozilla
make -f client.mk pull_all BUILD_MODULES=
env POSIXLY_CORRECT=1 patch -p0 < ~/standalone.patch
make -f client.mk pull_all BUILD_MODULES=xpcom
./configure --enable-modules=xpcom
make

Ok, so what am I actually doing here?  In build/unix/modules.mk, I'm creating a
list of dep dirs in the same way that I create BUILD_MODULE_DIRS.  Then I added
a new rule set to config/rules.mk that causes recursive makes to be triggered if
EXPORT_DIRS is set and it is only set in the toplevel Makefile.  By adding intl
& modules/libjar to the dep list, I am able to build xpcom and run nsTestSample
without actually building the intl nor libjar components.

Side notes:
Right now, it's still kinda lame that we still require the module deps to be
kept in 2 places (modules.mk & allmakefiles.sh).  Eventually, --enable-module
should be renamed to something like --enable-standalone.

Comment 6

[Peter "jag" Annema]

Looks good, r=jag

Comment 7

[cls]

Patch has been checked in.

Comment 8

[cls]

Attachment: Checkout specific subdirs instead of toplevel dirs

Comment 9

[cls]

Attachment 21318 [details] [diff] contains changes to make building standalone more efficient. 
Instead of pulling a toplevel directory like xpcom or intl (which are the entry
points for building those modules) for standalone builds, we will actually list
the directories we need to pull.  Since I was unable to find a way to duplicate
the exclude feature of cvs modules (!mozilla/dir), I modified client.mk so that
it will pull a specified list of directories using the no-recurse option (-l) so
that we recursively pull directories that we do not need.  For example, when
building js standalone, we do not need js/src/perlconnect, js/src/liveconnect,
etc, so we just add js js/src & js/src/fdlibm to the no-recurse pull list.  The
downside of this is that build maintainers have to be more vigilant about
tracking intermodule dependencies or else the standalone builds will be prone to
breakage.

Comment 10

[Axel Hecht]

Those checkins broke the transformiix standalone build :-(.
We need a _CVS_ in modules.mk, too. Adding this brings up two other bugs. They
are fixed in my tree now. BUILD_MODULE_CVS_NS and BUILD_MODULE_DEP_DIRS can be 
empty, the checks are either missing or wrong (as they are defined).
Patch follows in a second.

Axel

Comment 11

[Axel Hecht]

Attachment: fix for empty vars, transformiix standalone

Comment 12

[cls]

Sorry about breaking transformiix.  r=cls

Comment 13

[Axel Hecht]

fix for simple modules checked in

Axel

Comment 14

[Asa Dotzler [:asa]]

Bugs targeted at mozilla1.0 without the mozilla1.0 keyword moved to mozilla1.0.1 
(you can query for this string to delete spam or retrieve the list of bugs I've 
moved)

Comment 15

[Asa Dotzler [:asa]]

don't move bugs that are in the 1.0 dependency tree. sorry.

Comment 16

[hacker formerly known as [e-mail]]

This should be fixed by the checkin for bug 173066.  modules.mk is generated
using the module dependencies listed in Makefiles by REQUIRES. See
http://lxr.mozilla.org/mozilla/source/tools/module-deps/bootstrap.pl .  The full
list of modules (currently 147) can be found at the generated modules.mk.  See
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/build/unix/modules.mk#45


*** This bug has been marked as a duplicate of 173066 ***

Comment 17

[Tracy Walker [:tracy]]

verified dupe