783357 - use PARALLEL_DIRS in dom/

Status: RESOLVED FIXED

Description

[Nathan Froyd [:froydnj]]

Attachment: PARALLEL_DIRS magic for dom/

dom/ has grown pretty big, almost as big as content/.  content/ does PARALLEL_DIRS.  dom/ doesn't.  Let's fix that.

Fresh build before PARALLEL_DIRS:

real	10m46.380s
user	75m31.311s
sys	6m47.187s

Fresh build with attached patch:

real	10m9.947s
user	77m9.975s
sys	6m52.453s

so ~5% improvement.

Guessing there are some subtle ordering issues which one run doesn't turn up, though...

Comment 1

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

I'm pretty sure we can't do interfaces/ in parallel.

Comment 2

[Nathan Froyd [:froydnj]]

Yeah, I do see some random build failures on try.  May look at those today.

Comment 3

[Nathan Froyd [:froydnj]]

(In reply to :Ms2ger from comment #1)
> I'm pretty sure we can't do interfaces/ in parallel.

So it *looks* like the only thing preventing this from happening is that, e.g. domstubs.idl is included by a bunch of different files, thus requiring interfaces/base to be built before everything else.  (And other examples, but domstubs is the most prominent one that I've seen.)

dom/interfaces/base/Makefile.in gets around some circularity by appending to XPIDL_FLAGS; would it be reasonable to add the necessary XPIDL_FLAGS+= bits to the appropriate directories so it at least *looks* like we are declaring our dependencies, rather than requiring serialization to sort them out for us?

Comment 4

[Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary)]

fiddling with XPIDL_FLAGS (like LOCAL_INCLUDES) is the right way to proceed.

Comment 5

[Nathan Froyd [:froydnj]]

Attachment: add XPIDL_FLAGS where necessary

Here's the boring patch that adds XPIDL_FLAGS where necessary.  This whole patch was generated by a script, the logic is something like:

for each directory with idl files:
  determine #includes
  find where the #includes live
  write appropriate XPIDL_FLAGS bits to the directory's Makefile.in

so I am reasonably confident it won't produce random build failures.  I assumed that only dependencies in dom/ mattered; other deps should be taken care of by top-level serialization, IIUC.

Comment 6

[Nathan Froyd [:froydnj]]

Attachment: PARALLEL_DIRS magic for dom/

This patch flips dom/Makefile.in to use PARALLEL_DIRS in all the appropriate places.

Comment 7

[Nathan Froyd [:froydnj]]

CC'ing gps in case he has insights on this from build-splendid work.

Comment 8

[Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary)]

Comment on attachment 654617 [details] [diff] [review]
add XPIDL_FLAGS where necessary

Review of attachment 654617 [details] [diff] [review]:
-----------------------------------------------------------------

XPIDL_FLAGS += \
  directory \
  $(NULL)

Even on the single line ones please.

Comment 9

[Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary)]

Comment on attachment 654618 [details] [diff] [review]
PARALLEL_DIRS magic for dom/

Review of attachment 654618 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/Makefile.in
@@ +43,2 @@
>    interfaces/apps \
>    $(NULL)

Just move this up into the other set of PARALLEL_DIRS.

Comment 10

[Nathan Froyd [:froydnj]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/fb4219f3723d
https://hg.mozilla.org/integration/mozilla-inbound/rev/544b0cbdbb09

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/fb4219f3723d
https://hg.mozilla.org/mozilla-central/rev/544b0cbdbb09

Comment 12

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

So how about we back this out until we can do it right?

Comment 13

[(not currently active) Ted Mielczarek]

Yes, please back this out until we can fix the build issue.

Comment 14

[Nathan Froyd [:froydnj]]

FWIW, there is a patch for the build issue in bug 786703.