Last Comment Bug 783357 - use PARALLEL_DIRS in dom/
: use PARALLEL_DIRS in dom/
Product: Core
Classification: Components
Component: DOM (show other bugs)
: unspecified
: All All
: -- normal (vote)
: mozilla17
Assigned To: Nathan Froyd [:froydnj]
Depends on:
Blocks: 786703
  Show dependency treegraph
Reported: 2012-08-16 13:01 PDT by Nathan Froyd [:froydnj]
Modified: 2012-09-06 12:04 PDT (History)
4 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---

PARALLEL_DIRS magic for dom/ (483 bytes, patch)
2012-08-16 13:01 PDT, Nathan Froyd [:froydnj]
no flags Details | Diff | Splinter Review
add XPIDL_FLAGS where necessary (17.86 KB, patch)
2012-08-23 07:57 PDT, Nathan Froyd [:froydnj]
khuey: review+
Details | Diff | Splinter Review
PARALLEL_DIRS magic for dom/ (1.43 KB, patch)
2012-08-23 07:58 PDT, Nathan Froyd [:froydnj]
khuey: review+
Details | Diff | Splinter Review

Description Nathan Froyd [:froydnj] 2012-08-16 13:01:58 PDT
Created attachment 652529 [details] [diff] [review]
PARALLEL_DIRS magic for dom/

dom/ has grown pretty big, almost as big as content/.  content/ does PARALLEL_DIRS.  dom/ doesn't.  Let's fix that.

Fresh build before PARALLEL_DIRS:

real	10m46.380s
user	75m31.311s
sys	6m47.187s

Fresh build with attached patch:

real	10m9.947s
user	77m9.975s
sys	6m52.453s

so ~5% improvement.

Guessing there are some subtle ordering issues which one run doesn't turn up, though...
Comment 1 :Ms2ger (⌚ UTC+1/+2) 2012-08-17 01:04:58 PDT
I'm pretty sure we can't do interfaces/ in parallel.
Comment 2 Nathan Froyd [:froydnj] 2012-08-17 05:07:46 PDT
Yeah, I do see some random build failures on try.  May look at those today.
Comment 3 Nathan Froyd [:froydnj] 2012-08-22 14:06:39 PDT
(In reply to :Ms2ger from comment #1)
> I'm pretty sure we can't do interfaces/ in parallel.

So it *looks* like the only thing preventing this from happening is that, e.g. domstubs.idl is included by a bunch of different files, thus requiring interfaces/base to be built before everything else.  (And other examples, but domstubs is the most prominent one that I've seen.)

dom/interfaces/base/ gets around some circularity by appending to XPIDL_FLAGS; would it be reasonable to add the necessary XPIDL_FLAGS+= bits to the appropriate directories so it at least *looks* like we are declaring our dependencies, rather than requiring serialization to sort them out for us?
Comment 4 Kyle Huey [:khuey] ( 2012-08-22 14:15:32 PDT
fiddling with XPIDL_FLAGS (like LOCAL_INCLUDES) is the right way to proceed.
Comment 5 Nathan Froyd [:froydnj] 2012-08-23 07:57:34 PDT
Created attachment 654617 [details] [diff] [review]
add XPIDL_FLAGS where necessary

Here's the boring patch that adds XPIDL_FLAGS where necessary.  This whole patch was generated by a script, the logic is something like:

for each directory with idl files:
  determine #includes
  find where the #includes live
  write appropriate XPIDL_FLAGS bits to the directory's

so I am reasonably confident it won't produce random build failures.  I assumed that only dependencies in dom/ mattered; other deps should be taken care of by top-level serialization, IIUC.
Comment 6 Nathan Froyd [:froydnj] 2012-08-23 07:58:26 PDT
Created attachment 654618 [details] [diff] [review]
PARALLEL_DIRS magic for dom/

This patch flips dom/ to use PARALLEL_DIRS in all the appropriate places.
Comment 7 Nathan Froyd [:froydnj] 2012-08-23 10:51:10 PDT
CC'ing gps in case he has insights on this from build-splendid work.
Comment 8 Kyle Huey [:khuey] ( 2012-08-24 12:23:53 PDT
Comment on attachment 654617 [details] [diff] [review]
add XPIDL_FLAGS where necessary

Review of attachment 654617 [details] [diff] [review]:

  directory \

Even on the single line ones please.
Comment 9 Kyle Huey [:khuey] ( 2012-08-24 12:24:35 PDT
Comment on attachment 654618 [details] [diff] [review]
PARALLEL_DIRS magic for dom/

Review of attachment 654618 [details] [diff] [review]:

::: dom/
@@ +43,2 @@
>    interfaces/apps \
>    $(NULL)

Just move this up into the other set of PARALLEL_DIRS.
Comment 12 :Ms2ger (⌚ UTC+1/+2) 2012-09-06 08:30:36 PDT
So how about we back this out until we can do it right?
Comment 13 Ted Mielczarek [:ted.mielczarek] 2012-09-06 09:02:20 PDT
Yes, please back this out until we can fix the build issue.
Comment 14 Nathan Froyd [:froydnj] 2012-09-06 12:04:34 PDT
FWIW, there is a patch for the build issue in bug 786703.

Note You need to log in before you can comment on or make changes to this bug.