469355 - frequent hangs of "win2k3 comm-central sunbird" buildbot [Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process]

Status: RESOLVED FIXED

(Calendar :: Build Config, defect)

Description

[Stefan Sitter [:ssitter]]

The "win2k3 comm-central sunbird nightly" build fails frequently with the error message: "Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process"

This happened 2008/12/12, 2008/12/11, 2008/12/08, 2008/12/02, 2008/12/01, 2008/11/30, 2008/11/27, 2008/11/15, ...

Often this was caused by a bunch of stuck make.exe processes. After killing the processes and restarting the buildbot client this issue is fixed for some days.

Maybe this is related to Sunbird doing parallel builds because of the -j6 make flags in the mozconfig. Thunderbird and Lightning don't do parallel builds and as far as I know the error did not showed up on that buildbots.

Comment 1

[ause]

Attachment: [checked in] disable multiprocess building on windows

yes, i could reproduce that locally too. not sure if that's a msys gnu-make or a makefile problem.

Comment 2

[Stefan Sitter [:ssitter]]

Comment on attachment 353038 [details] [diff] [review]
[checked in] disable multiprocess building on windows

Please remove the line instead of commenting out
r=ssitter

Comment 3

[Justin Wood (:Callek)]

I occassionaly (rarely) get this building TB and SM as well... any chance we can get/find a real bug on this in/under build-config... I'd love to figure out why.

(I suspect it has _something_ to do with our Parallel build support stuff)

Comment 4

[Stefan Sitter [:ssitter]]

(In reply to comment #3)
See Bug 381004, Bug 420216

Comment 5

[ause]

what i observed so far:
the SIGKILL message is, as already mentioned, a symptom of the hanging make. also this seems to happen only on clean builds. incremental builds, afaik, never had that problem.
what i suspect are two sub make calls in the same subdir, triggered by different targets, that mess up makes job management.
is there any tooling that can do a nice directory graph out of the "make -d" output?