286179 - make parallel builds possible on Windows using MSVC

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Chase Phillips]

Figure out why distcc and parallel builds fail on Windows.  My last try to make
this work was on chroma.  The build error log is at:

 
http://tinderbox.mozilla.org/showlog.cgi?log=Mozilla-l10n/1110855013.8962.gz&fulltext=1

At the time, I was running distccd on sweetlou.

Comment 1

[cls]

Parallel builds fail on MSVC builds because certain versions of MSVC write debug
information to a .pdb file per directory.  During a parallel build, 2 instances
of MSVC are attempting to write to the same file which doesn't work.  gcc builds
do not have this problem.

I thought newer versions of MSVC used a different debugging mechanism so that
this was no longer a problem?

Comment 2

[Chase Phillips]

(In reply to comment #1)
> Parallel builds fail on MSVC builds because certain versions of MSVC write debug
> information to a .pdb file per directory.  During a parallel build, 2 instances
> of MSVC are attempting to write to the same file which doesn't work.  gcc builds
> do not have this problem.

Hmm, from config/rules.mk the code appears to use $LIBRARY_NAME.pdb if
$LIBRARY_NAME is defined and <filename>.pdb otherwise.  This code is only
invoked on Windows if the compiler isn't gcc.  Can someone confirm this?

http://lxr.mozilla.org/mozilla/source/config/rules.mk#175

I'm tempted to tender a patch that removes the $LIBRARY_NAME conditional and
dump debug information into <filename>.pdb files across the board instead.  What
reasons were there to dump debugging information from multiple files into a
single .pdb file?  Are they still valid?

> I thought newer versions of MSVC used a different debugging mechanism so that
> this was no longer a problem?

I'm not certain what MSVC versions you mean when you say "newer" but we're still
using MSVC 6 for builds in-house.

Comment 3

[cls]

The /PDB:file rules came from the legacy Netscape nmake build system so I'm not
sure what the justification for a single .pdb file is.  Can you pass multiple
.pdb files to the linker?  Yes, the .pdb files are not used by gcc.

I thought that bryner was doing some work to support debugging using something
other than .pdb files for MSVC7.1 but I could just be misremembering an old bug.

Comment 4

[Chase Phillips]

Attachment: write debug symbols into separate pdb files - v 1 [checked in]

Comment 5

[Chase Phillips]

Comment on attachment 182988 [details] [diff] [review]
write debug symbols into separate pdb files - v 1 [checked in]

I'll work with Jay to monitor symbol generation and Talkback.  At some point we
should have enough crash data on 1.1a1 to be certain that keeping this fix is
the right thing to do.

Comment 6

[Chase Phillips]

Comment on attachment 182988 [details] [diff] [review]
write debug symbols into separate pdb files - v 1 [checked in]

a=chase

Comment 7

[Chase Phillips]

Comment on attachment 182988 [details] [diff] [review]
write debug symbols into separate pdb files - v 1 [checked in]

Landed this on the trunk plus an accompanying comment to make intent clear.

Checking in rules.mk;
/cvsroot/mozilla/config/rules.mk,v  <--  rules.mk
new revision: 3.475; previous revision: 3.474
done

Comment 8

[[e-mail]]

Attachment: missing link

Comment 9

[[e-mail]]

Comment on attachment 242403 [details] [diff] [review]
missing link

MSVC debug build config only, no effect on releases.

Comment 10

[Benjamin Smedberg]

Not for 1.8.1. Let's have this bake for a week on trunk and we can pick it up for 1.8.1.1

Comment 11

[Mike Beltzner [:beltzner, not reading bugmail]]

Comment on attachment 242403 [details] [diff] [review]
missing link

as per bsmedberg's comment

Comment 12

[Daniel Veditz [:dveditz]]

Comment on attachment 242403 [details] [diff] [review]
missing link

approved for 1.8 branch, a=dveditz for drivers

Comment 13

[[e-mail]]

Fix checked in to the branch.