490389 - Mac bloat builds dying trying to run fix-macosx-stack.pl

Status: RESOLVED FIXED

(Mozilla Messaging Graveyard :: Release Engineering, defect)

Description

[Phil Ringnalda (:philor)]

Since last night, around the time of the bug 490266 clobber and the appearance of momo-vm-osx-tiger-01 doing bloat builds, the

/bin/bash -c perl build/mozilla/tools/rb/fix-macosx-stack.pl sdleak.tree.raw > sdleak.tree

step has been dying  with

command timed out: 3600 seconds without output, killing pid 25817
process killed by signal 9
program finished with exit code -1

on both momo-vm-osx-tiger-01 and momo-vm-09.

Comment 1

[Philippe M. Chiasson (:gozer)]

For some reason, that script is very heavy on IO/CPU and produces no output, so it's all about guessing a reasonable timeout value for buildbot. 60 minutes so far seems like it had been enough, but not anymore.

Not entirely sure why this is needing more time to complete, but I am reluctant to just increase the timeout value. Ideally, I'd like to dive into why that script is so expensive to run, and see if it can be optimized some.

In the meantime, one possible solution is just to ensure output is generated by that script. Right now, I made a small local tweak to buildbot to run

perl [...]fix-macosx-stack.pl sdleak.tree.raw | tee sdleak.tree

instead, so the continuous output should keep buildbot from timing out, at the cost of increasing the size of the logs (not sure by home much).

Let's see how that goes.

Comment 2

[Philippe M. Chiasson (:gozer)]

Hasn't hapenned again, closing for now.