909478 - deploy a distcc machine for linux and mac builds in MV

Status: NEW

(Core :: General, defect)

Description

[Andreas Gal :gal]

A lot of people are using linux and mac for development. We should see whether distcc already makes builds a bit faster, and what we have to do to make it even faster than that. It would be particularly useful if a Linux box can do this for mac and linux, but buying mac hardware is fine as well.

Comment 1

[(no longer active)]

distcc doesn't make our builds very fast unless if you're on very slow hardware (like a macbook air, for example) because we currently waste a lot of time waiting on make invocations to finish.  It will, however, when we have a non-recursive build system.

Comment 2

[Boris Zbarsky [:bzbarsky]]

Fwiw, it used to be possible to do Mac builds on Linux via.  See http://weblogs.mozillazine.org/bz/archives/2010/01/crosscompiling_success.html and I believe I got that working with distcc as well.  Note that at the time I had to fix a bunch of build system bugs to get this working; it may be broken again now.

http://weblogs.mozillazine.org/bz/archives/020371.html had some data from back then on performance of the distcc setup in various configurations too.

Comment 3

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

We plan on experimenting with distcc with the ninja backend, once a few more kinks are worked out.  Because it's non-recursive it can give us a good idea of the possible speedup, especially in the simpler mode where the local host does the preprocessing (and thus might be bound by it).

Comment 4

[Boris Zbarsky [:bzbarsky]]

Note that doing preprocessing on the local host also lets you use ccache on the local host, so it's possibly preferable anyway.

Comment 5

[Mike Hommey [:glandium]]

FWIW, last time i tried distcc using two machines was slower than a local build on the beefier of the two.

Note that doing preprocessing on the remote host would allow to use ccache on the remote host. If the remote host is used by different people, that makes a shared ccache. Not sure that would be a win, though.

Comment 6

[John Schoenick [:johns]]

We have a few machines linked up to do distcc, and it definitely provides a speed increase over local compilation. Using pump mode I can saturate four machines with compile jobs for parts of the build like dom/ and content/. Not using pump mode gives a slightly less impressive speedup, as you quickly become bottlenecked on preprocessing.

(In reply to Mike Hommey [:glandium] from comment #5)
> Note that doing preprocessing on the remote host would allow to use ccache
> on the remote host. If the remote host is used by different people, that
> makes a shared ccache. Not sure that would be a win, though.

Are you sure this would work? IIRC pump mode re-creates the transient files on the remote side, so will never get ccache hits.

Comment 7

[John Schoenick [:johns]]

Also, with non-pump mode on OS X you can make use of linux distcc nodes that have a compatible clang version:
>  export CC="clang -target x86_64-apple-darwin"
>  export CXX="clang++ -target x86_64-apple-darwin"