Closed
Bug 463385
Opened 16 years ago
Closed 14 years ago
sprinkle magic PARALLEL_DIRS fairy dust about the build system
Categories
(MailNews Core :: Build Config, defect)
MailNews Core
Build Config
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: neil, Assigned: neil)
References
Details
Attachments
(2 files, 4 obsolete files)
28.04 KB,
patch
|
Details | Diff | Splinter Review | |
28.06 KB,
patch
|
gozer
:
review+
Bienvenu
:
superreview+
|
Details | Diff | Splinter Review |
mailnews/ version of bug 462381
Assignee | ||
Comment 1•16 years ago
|
||
I also bypassed some makefiles which only one one child in DIRS.
Assignee: nobody → neil
Status: NEW → ASSIGNED
Attachment #346650 -
Flags: superreview?(bienvenu)
Attachment #346650 -
Flags: review?(gozer)
Comment 2•16 years ago
|
||
Comment on attachment 346650 [details] [diff] [review] Proposed patch I'll file a follow up bug for the TB-specific dirs
Attachment #346650 -
Flags: superreview?(bienvenu) → superreview+
Comment 3•16 years ago
|
||
Comment on attachment 346650 [details] [diff] [review] Proposed patch a full clobber build with -j8 and this patch breaks for me on Linux 4 times before finally working. Summary of the errors are : gmake[6]: Leaving directory `/home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mailnews/base/test' gmake[6]: Leaving directory `/home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mailnews/base/search/src' nsICopyMessageListener.idl ../../../mozilla/dist/bin/xpidl -m header -w -I/home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/public -I../../../mozilla/dist/idl -o _xpidlgen/nsICopyMessageListener /home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/public/nsICopyMessageListener.idl Creating .deps /home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mozilla/config/nsinstall -D ../../../../mozilla/dist/include/msgbase /home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mozilla/config/nsinstall -D ../../../../mozilla/dist/idl gmake[6]: Leaving directory `/home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mailnews/base/src' Creating _xpidlgen/.done /home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mozilla/config/nsinstall -D ../../../../mozilla/dist/idl nsIMsgFilter.idl ../../../../mozilla/dist/bin/xpidl -m header -w -I/home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/search/public -I../../../../mozilla/dist/idl -o _xpidlgen/nsIMsgFilter /home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/search/public/nsIMsgFilter.idl /home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/search/public/nsMsgFilterCore.idl:39: can't open included file MailNewsTypes2.idl for reading input callback returned failure gmake[6]: *** [_xpidlgen/nsIMsgFilter.h] Error 2 gmake[6]: Leaving directory `/home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mailnews/base/search/public' nsICopyMsgStreamListener.idl ../../../mozilla/dist/bin/xpidl -m header -w -I/home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/public -I../../../mozilla/dist/idl -o _xpidlgen/nsICopyMsgStreamListener /home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/public/nsICopyMsgStreamListener.idl gmake[5]: *** [search/public_export] Error 2 gmake[5]: *** Waiting for unfinished jobs.... [...] c++ -o nsMsgBodyHandler.o -c -I../../../../mozilla/dist/include/system_wrappers -include /home/gozer/opt/src/mozilla.org/comm-central/mozilla/config/gcc_hidden.h -DMOZILLA_INTERNAL_API -DMOZ_THUNDERBIRD=1 -DOSTYPE=\"Linux2.6.26.6-49\" -DOSARCH=Linux -I/home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/search/src -I. -I../../../../mozilla/dist/include/xpcom -I../../../../mozilla/dist/include/string -I../../../../mozilla/dist/include/msgdb -I../../../../mozilla/dist/include/msgbaseutil -I../../../../mozilla/dist/include/necko -I../../../../mozilla/dist/include/mime -I../../../../mozilla/dist/include/msgimap -I../../../../mozilla/dist/include/rdf -I../../../../mozilla/dist/include/msglocal -I../../../../mozilla/dist/include/intl -I../../../../mozilla/dist/include/unicharutil -I../../../../mozilla/dist/include/locale -I../../../../mozilla/dist/include/msgcompose -I../../../../mozilla/dist/include/pref -I../../../../mozilla/dist/include/msgnews -I../../../../mozilla/dist/include/txmgr -I../../../../mozilla/dist/include/addrbook -I../../../../mozilla/dist/include/docshell -I../../../../mozilla/dist/include -I../../../../mozilla/dist/include/msgbase `../../../../mozilla/nsprpub/config/nspr-config --prefix=../../../../mozilla/dist --includedir=../../../../mozilla/dist/include/nspr --cflags` -fPIC -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-long-long -pedantic -gstabs+ -fno-strict-aliasing -fshort-wchar -pthread -pipe -DDEBUG -D_DEBUG -DDEBUG_gozer -DTRACING -g -fno-inline -DMOZILLA_CLIENT -include ../../../../mozilla-config.h -Wp,-MD,.deps/nsMsgBodyHandler.pp /home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/search/src/nsMsgBodyHandler.cpp c++ -o nsMimeObjectClassAccess.o -c -I../../../mozilla/dist/include/system_wrappers -include /home/gozer/opt/src/mozilla.org/comm-central/mozilla/config/gcc_hidden.h -DENABLE_SMIME -DMOZILLA_INTERNAL_API -DMOZ_THUNDERBIRD=1 -DOSTYPE=\"Linux2.6.26.6-49\" -DOSARCH=Linux -I/home/gozer/opt/src/mozilla.org/comm-central/mailnews/mime/src -I. -I../../../mozilla/dist/include/xpcom -I../../../mozilla/dist/include/string -I../../../mozilla/dist/include/msgbase -I../../../mozilla/dist/include/msgbaseutil -I../../../mozilla/dist/include/pref -I../../../mozilla/dist/include/js -I../../../mozilla/dist/include/necko -I../../../mozilla/dist/include/uconv -I../../../mozilla/dist/include/unicharutil -I../../../mozilla/dist/include/msgcompose -I../../../mozilla/dist/include/dom -I../../../mozilla/dist/include/intl -I../../../mozilla/dist/include/msgimap -I../../../mozilla/dist/include/exthandler -I../../../mozilla/dist/include/chardet -I../../../mozilla/dist/include/caps -I../../../mozilla/dist/include/mimetype -I../../../mozilla/dist/include/xpconnect -I../../../mozilla/dist/include/nkcache -I../../../mozilla/dist/include/pipnss -I../../../mozilla/dist/include/imglib2 -I../../../mozilla/dist/include/content -I../../../mozilla/dist/include/htmlparser -I../../../mozilla/dist/include/layout -I../../../mozilla/dist/include/msgsmime -I../../../mozilla/dist/include -I../../../mozilla/dist/include/mime `../../../mozilla/nsprpub/config/nspr-config --prefix=../../../mozilla/dist --includedir=../../../mozilla/dist/include/nspr --cflags` -fPIC -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-long-long -pedantic -gstabs+ -fno-strict-aliasing -fshort-wchar -pthread -pipe -DDEBUG -D_DEBUG -DDEBUG_gozer -DTRACING -g -fno-inline -DMOZILLA_CLIENT -include ../../../mozilla-config.h -Wp,-MD,.deps/nsMimeObjectClassAccess.pp /home/gozer/opt/src/mozilla.org/comm-central/mailnews/mime/src/nsMimeObjectClassAccess.cpp gmake[6]: Entering directory `/home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mailnews/db/msgdb/build' gmake[6]: *** No rule to make target `-lmsgbaseutil', needed by `libmsgdb.so'. Stop. gmake[6]: *** Waiting for unfinished jobs.... [...] c++ -o nsMsgSend.o -c -I../../../mozilla/dist/include/system_wrappers -include /home/gozer/opt/src/mozilla.org/comm-central/mozilla/config/gcc_hidden.h -DMOZ_APP_VERSION="3.0b1pre" -DMOZILLA_INTERNAL_API -DMOZ_THUNDERBIRD=1 -DOSTYPE=\"Linux2.6.26.6-49\" -DOSARCH=Linux -I/home/gozer/opt/src/mozilla.org/comm-central/mailnews/compose/src -I. -I../../../mozilla/dist/include/xpcom -I../../../mozilla/dist/include/string -I../../../mozilla/dist/include/dom -I../../../mozilla/dist/include/layout -I../../../mozilla/dist/include/editor -I../../../mozilla/dist/include/uriloader -I../../../mozilla/dist/include/msgbase -I../../../mozilla/dist/include/necko -I../../../mozilla/dist/include/mime -I../../../mozilla/dist/include/pref -I../../../mozilla/dist/include/intl -I../../../mozilla/dist/include/lwbrk -I../../../mozilla/dist/include/uconv -I../../../mozilla/dist/include/locale -I../../../mozilla/dist/include/unicharutil -I../../../mozilla/dist/include/content -I../../../mozilla/dist/include/msgbaseutil -I../../../mozilla/dist/include/msgdb -I../../../mozilla/dist/include/rdf -I../../../mozilla/dist/include/appshell -I../../../mozilla/dist/include/toolkitcomps -I../../../mozilla/dist/include/appcomps -I../../../mozilla/dist/include/msgimap -I../../../mozilla/dist/include/msgnews -I../../../mozilla/dist/include/docshell -I../../../mozilla/dist/include/addrbook -I../../../mozilla/dist/include/widget -I../../../mozilla/dist/include/htmlparser -I../../../mozilla/dist/include/txmgr -I../../../mozilla/dist/include/caps -I../../../mozilla/dist/include/exthandler -I../../../mozilla/dist/include/nkcache -I../../../mozilla/dist/include/mimetype -I../../../mozilla/dist/include/windowwatcher -I../../../mozilla/dist/include/gfx -I../../../mozilla/dist/include/zipwriter -I../../../mozilla/dist/include/xulapp -I../../../mozilla/dist/include -I../../../mozilla/dist/include/msgcompose `../../../mozilla/nsprpub/config/nspr-config --prefix=../../../mozilla/dist --includedir=../../../mozilla/dist/include/nspr --cflags` -fPIC -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-long-long -pedantic -gstabs+ -fno-strict-aliasing -fshort-wchar -pthread -pipe -DDEBUG -D_DEBUG -DDEBUG_gozer -DTRACING -g -fno-inline -DMOZILLA_CLIENT -include ../../../mozilla-config.h -Wp,-MD,.deps/nsMsgSend.pp /home/gozer/opt/src/mozilla.org/comm-central/mailnews/compose/src/nsMsgSend.cpp chmod +x libmsgmdn.so /home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mozilla/config/nsinstall -R -m 755 libmsgmdn.so ../../../../mozilla/dist/bin/components : ../../../../mozilla/dist/bin/components/libmsgmdn.so gmake[7]: Leaving directory `/home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mailnews/extensions/mdn/build' c++: ../src/libmsgimap_s.a: No such file or directory gmake[6]: *** [TestImapHdrXferInfo] Error 1 gmake[6]: Leaving directory `/home/gozer/opt/src/mozilla.org/objdir-x86_64-unknown-linux-gnu-bloat/mailnews/imap/test' gmake[5]: *** [test_libs] Error 2 gmake[5]: *** Waiting for unfinished jobs.... [...]
Attachment #346650 -
Flags: review?(gozer) → review-
Assignee | ||
Comment 4•16 years ago
|
||
* mailnews/base can't be parallelised, so make it explicitly in build.mk * imap/test can't be parallelised
Attachment #346650 -
Attachment is obsolete: true
Attachment #346897 -
Flags: superreview?(bienvenu)
Attachment #346897 -
Flags: review?(gozer)
Comment 5•16 years ago
|
||
Comment on attachment 346897 [details] [diff] [review] Revised patch Still something broken, with a full clobber and make -j8, I get: gmake[4]: Leaving directory `[objdir]/mail/branding/nightly' gmake[4]: Entering directory `[objdir]/mailnews/base' gmake[4]: *** No rule to make target `[comm-central]/mailnews/base/search/Makefile.in', needed by `search/Makefile'. Stop. gmake[4]: Leaving directory `[objdir]/mailnews/base' gmake[3]: *** [export_tier_app] Error 2 gmake[3]: Leaving directory `[objdir]' gmake[2]: *** [tier_app] Error 2 gmake[2]: Leaving directory `[objdir]' make[1]: *** [default] Error 2 make[1]: Leaving directory `[objdir]' make: *** [build] Error 2
Comment 6•16 years ago
|
||
Ah, my bad, I didn't apply that patch to the tip, disregard my comment #5, sorry.
Comment 7•16 years ago
|
||
Actually, my double-bad. I've tried this patch against the tip, and I get the same failure. Seems something is depending on the now removed mailnews/base/search/Makefile.in
Assignee | ||
Comment 8•16 years ago
|
||
The diff to mailnews/base/Makefile.in is the same as in the original patch.
Attachment #346897 -
Attachment is obsolete: true
Attachment #346916 -
Flags: superreview?(bienvenu)
Attachment #346916 -
Flags: review?(gozer)
Attachment #346897 -
Flags: superreview?(bienvenu)
Attachment #346897 -
Flags: review?(gozer)
Comment 9•16 years ago
|
||
Comment on attachment 346916 [details] [diff] [review] Restored patch Still failing on clobber & make -j8 ../../../../mozilla/dist/bin/xpidl -m header -w -I[comm-central]/mailnews/base/search/public -I../../../../mozilla/dist/idl -o _xpidlgen/nsIMsgFilter [comm-central]/mailnews/base/search/public/nsIMsgFilter.idl gmake[5]: Leaving directory `[objdir]/mailnews/base/src' nsIMsgFilterHitNotify.idl ../../../../mozilla/dist/bin/xpidl -m header -w -I[comm-central]/mailnews/base/search/public -I../../../../mozilla/dist/idl -o _xpidlgen/nsIMsgFilterHitNotify [comm-central]/mailnews/base/search/public/nsIMsgFilterHitNotify.idl nsIFolderListener.idl ../../../mozilla/dist/bin/xpidl -m header -w -I[comm-central]/mailnews/base/public -I../../../mozilla/dist/idl -o _xpidlgen/nsIFolderListener [comm-central]/mailnews/base/public/nsIFolderListener.idl [comm-central]/mailnews/base/search/public/nsMsgFilterCore.idl:39: can't open included file MailNewsTypes2.idl for reading input callback returned failure gmake[5]: *** [_xpidlgen/nsIMsgFilter.h] Error 2 gmake[5]: *** Waiting for unfinished jobs.... /usr/bin/gmake -C search/src export gmake[5]: Entering directory `[objdir]/mailnews/base/search/src' Creating .deps nsIFolderLookupService.idl ../../../mozilla/dist/bin/xpidl -m header -w -I[comm-central]/mailnews/base/public -I../../../mozilla/dist/idl -o _xpidlgen/nsIFolderLookupService [comm-central]/mailnews/base/public/nsIFolderLookupService.idl gmake[5]: Leaving directory `[objdir]/mailnews/base/search/src' /usr/bin/perl -I[comm-central]/mozilla/config [comm-central]/mozilla/config/build-list.pl ../../../mozilla/config/final-link-libs msgbaseutil [comm-central]/mailnews/base/search/public/nsMsgFilterCore.idl:39: can't open included file MailNewsTypes2.idl for reading input callback returned failure gmake[5]: *** [_xpidlgen/nsIMsgFilterHitNotify.h] Error 2 gmake[5]: Leaving directory `[objdir]/mailnews/base/search/public' gmake[4]: *** [search/public_export] Error 2 gmake[4]: *** Waiting for unfinished jobs....
Attachment #346916 -
Flags: review?(gozer) → review-
Assignee | ||
Comment 10•16 years ago
|
||
OK, that should be as easy as moving search/* back into DIRS In theory I could move base/search/* into mailnews/Makefile's PARALLEL_DIRS and base/build into mailnews/Makefile's DIRS for optimum parallelisation ;-)
Attachment #346916 -
Attachment is obsolete: true
Attachment #346936 -
Flags: superreview?(bienvenu)
Attachment #346936 -
Flags: review?(gozer)
Attachment #346916 -
Flags: superreview?(bienvenu)
Comment 11•16 years ago
|
||
(In reply to comment #10) > Created an attachment (id=346936) [details] >[...] > > In theory I could move base/search/* into mailnews/Makefile's PARALLEL_DIRS and > base/build into mailnews/Makefile's DIRS for optimum parallelisation ;-) Then why not just go ahead and do that ? Parallelism is good!
Assignee | ||
Comment 12•16 years ago
|
||
I won't cancel the previous reviews just in case ;-) I moved base/search/src into mailnews/Makefile's PARALLEL_DIRS so that it gets built after base/util, and base/build into mailnews/Makefile's DIRS so that it gets built after base/search/src, but I kept search/public in base/Makefile.
Attachment #346943 -
Flags: superreview?(bienvenu)
Attachment #346943 -
Flags: review?(gozer)
Comment 13•16 years ago
|
||
Comment on attachment 346943 [details] [diff] [review] Your wish is my command sr=me, as long as it works for gozer :-)
Attachment #346943 -
Flags: superreview?(bienvenu) → superreview+
Comment 14•16 years ago
|
||
Comment on attachment 346943 [details] [diff] [review] Your wish is my command /opt/src/mozilla.org/comm-central/mailnews/base/search/public/nsIMsgSearchTerm.idl /usr/bin/perl -I/home/gozer/opt/src/mozilla.org/comm-central/mozilla/config /home/gozer/opt/src/mozilla.org/comm-central/mozilla/config/build-list.pl ../../../mozilla/config/final-link-libs msgbaseutil /home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/search/public/nsMsgFilterCore.idl:39: can't open included file MailNewsTypes2.idl for reading input callback returned failure /home/gozer/opt/src/mozilla.org/comm-central/mailnews/base/search/public/nsMsgFilterCore.idl:39: can't open included file MailNewsTypes2.idl for reading input callback returned failure gmake[5]: *** [_xpidlgen/nsIMsgFilter.h] Error 2 gmake[5]: *** Waiting for unfinished jobs....
Attachment #346943 -
Flags: review?(gozer) → review-
Assignee | ||
Comment 15•16 years ago
|
||
It looks like search/public doesn't parallelise either. Fingers crossed ;-)
Attachment #346943 -
Attachment is obsolete: true
Attachment #347091 -
Flags: review?(gozer)
Updated•16 years ago
|
Attachment #347091 -
Flags: review?(gozer) → review+
Comment 16•16 years ago
|
||
Comment on attachment 347091 [details] [diff] [review] Slightly less parallel clobber + make -j8 worked for me.
Comment 17•16 years ago
|
||
Comment on attachment 347091 [details] [diff] [review] Slightly less parallel I'm going to take the liberty of moving the sr request to this patch
Attachment #347091 -
Flags: superreview+
Comment 18•16 years ago
|
||
If anyone runs into chrome problems due to incomplete .jar packaging in parallel builds, see bug 464080.
Comment 19•15 years ago
|
||
Comment on attachment 346936 [details] [diff] [review] Fourth time lucky? clearing sr request - I think this is obsolete if not, let me know...
Attachment #346936 -
Flags: superreview?(bienvenu) → superreview?
Comment 20•15 years ago
|
||
Comment on attachment 346936 [details] [diff] [review] Fourth time lucky? I believe these requests are obsolete, please re-request if necessary.
Attachment #346936 -
Flags: superreview?
Attachment #346936 -
Flags: review?(gozer)
Comment 21•14 years ago
|
||
Neil this bug hasn't been touched in a while; more to do or should we resolve this and do any additional cleanup in separate bugs?
Assignee | ||
Comment 22•14 years ago
|
||
Yes, I guess we can mark this fixed, and file followup bugs as necessary.
Status: ASSIGNED → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•