485559 - [meta] Better support for distributed compilation

Status: NEW

(Firefox Build System :: General, defect)

Description

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

For much of the day, my machine (and its virtual machine) sits essentially idle.  In addition, I have another machine I plan on using for private distcc.  Why should my dev machine be mostly idle, and my distcc machine be private, when they can both be used as tryserver/build slaves?

It'd be great to set up a distcc network encompassing, at least, the main MoCo office.  Participation could be optional, but strongly encouraged.  It'd also be worth investigating John Ousterhout's Electric Cloud (http://www.electric-cloud.com/products/electricaccelerator.php), which according to Prof. Ousterhout, can outperform distcc for build systems not specifically tuned for distcc.  It performs about the same as distcc for build systems that are tuned for distcc (like the linux kernel's).

Comment 1

[Benjamin Smedberg]

I thought there was a distcc farm for the MoCo office that vlad set up: vlad is that true?

Comment 2

[(not currently active) Ted Mielczarek]

This doesn't really belong in Core: Build Config, but I dunno where to put it. AFAIK Mozilla builds fine with distcc (although I have never personally tried it).

Comment 3

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to comment #2)
> This doesn't really belong in Core: Build Config, but I dunno where to put it.
> AFAIK Mozilla builds fine with distcc (although I have never personally tried
> it).

Just to be clear, by "tuned for distcc," I meant that the Makefiles are written so that distcc can best identify independent tasks.  I've never done this myself, but I understand that it can require some work.

Comment 4

[David Mandelin [:dmandelin]]

Chris, have you gotten distcc to build FF? The TraceMonkey team has an 8-core Mac with distcc server installed, but ISTR that my first effort to build FF that way failed. If you can figure out how to get it to work, that would be very cool.

Comment 5

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to comment #4)
> Chris, have you gotten distcc to build FF? The TraceMonkey team has an 8-core
> Mac with distcc server installed, but ISTR that my first effort to build FF
> that way failed. If you can figure out how to get it to work, that would be
> very cool.

No, I'm not yet able to try.  Waiting on a hub from IT.  I'll see if I can find a way around the lack of a hub, though.

Comment 6

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to comment #1)
> I thought there was a distcc farm for the MoCo office that vlad set up: vlad is
> that true?

Nope; I wish there was, though!  It would help with linux/OSX builds, but I don't think it would help win32 at all (even assuming that the win32 distcc issues were resolved) since the compiler there is so fast anyway.

Comment 7

[(not currently active) Ted Mielczarek]

There were some Seneca students that got distcc working on Win32:
http://zenit.senecac.on.ca/wiki/index.php/Distcc_With_MSVC
http://zenit.senecac.on.ca/wiki/index.php/Distcc_mozilla_tutorial

so I am skeptical that there are large issues with our build system.

Comment 8

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to comment #4)
> Chris, have you gotten distcc to build FF? The TraceMonkey team has an 8-core
> Mac with distcc server installed, but ISTR that my first effort to build FF
> that way failed. If you can figure out how to get it to work, that would be
> very cool.

What's the machine's address?  Can I get SSH access?  I want to start working on this.  (Might be a while yet still on my switch.)

Comment 9

[David Humphrey (:humph)]

I have not been able to get distcc builds working on OSX with the stuff shipped in Leopard, despite a bunch of time playing with it--the builds fail before they reach the end (I don't have a specific build error handy).  I have heard that it does work if you replace the apple distcc with the upstream one.

Comment 10

[Gregory Szorc [:gps]]

I'm going to re-purpose this bug to track an effort around better distributed compilation, both in the build system and in physical facilities - notably Mozilla offices.

Comment 11

[Gregory Szorc [:gps]]

I have contacted Corey Shields in IT regarding what it will take to set up a distributed compilation service in Mozilla offices.