330779 - Tinderbox is extraordinarly bad at killing timed-out child processes

Status: RESOLVED WONTFIX

(Webtools Graveyard :: Tinderbox, defect)

Description

[Mark Mentovai]

As reported in bug 330763 and maybe elsewhere, and as known by anyone who's ever set up a tinderbox before:

Tinderbox is absolutely terrible about letting child process run amok.

There are a few things in tinderbox that run as processes wrapped up in timeouts, but when the processes time out, tinderbox doesn't kill them.  Oh, it says it kills them, all right, but it's lying.  cvs is the prime example.  The CVS server takes a nap, and a cvs checkout hangs for a while.  You hit the timeout (usually 1h).  The log says "Process killed. Took 1 second to die."  Great.  But when you run ps, you see that you've got all of these dopey cvs processes hanging around.  What happened?

I know!

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/tools/tinderbox/build-seamonkey-util.pl&rev=1.313&mark=599,603,608#590

"Process killed. Took 1 second to die" refers to the forked perl process (child), not to the subprocess that it spawned to do any of the hard work (grandchild).  In some cases, the subprocess will be a shell invocation, and the hard work is actually being done by the great-grandchild.

Oh, boy, this needs to be cleaned up!

Comment 1

[Mark Mentovai]

[11:48pm] * mento hugs tinderbox

Comment 2

[cls]

http://www.seawood.org/blog/archives/2006/10/putting_cvs_on_timeout.html

Comment 3

[J. Paul Reed [:preed]]

See also:

http://lxr.mozilla.org/mozilla/source/tools/release/MozBuild/Util.pm#19

which, from my very cursory reading of the code in the blog post above, does pretty much the same thing.

As bsmedberg and I were discussing this morning, it probably won't work on Windows the way you'd expect.

We're already starting the process of helping Tinderbox use RunShellCommand() over the 48 different (and most of them, broken somehow) ways it calls subcommands.

Comment 4

[Sylvestre Ledru [:Sylvestre]]

Tinderbox isn't maintained anymore. Closing.