1428509 - Log something more useful than "pipe error: Connection reset by peer" when a child process exits

Status: NEW

(Core :: IPC, enhancement, P3)

Description

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

We keep getting bug reports where child processes crash (or exit somehow) and the only related error message is the "pipe error (%d): Connection reset by peer" from IPC, which tells us nothing about why the process exited, and makes people assume IPC is involved in the bug even though it could be unrelated.

There may not be a good way to avoid that in the case of an unexpected exit, and it might not even be easy to avoid this on normal shutdown, but one thing we could do that might help is log the actual exit status (at least if it's a failure, but if there's "pipe error" spam in some cases of successful exit then also there).

Comment 1

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Bug 1427172 had a crash report from the child process in question, but there was a “crash” (SIGABRT due to timeout) that was processed first, so someone viewing the log would have to think to scroll all the way past it to look for the real crash.  A log message like “process NNNNN killed by SIGSEGV” next to (or in place of) the pipe error could improve the user experience, even though the information is already present in the log file.

Bug 1430065 is a parent process crash; in that case it's the child that complains about having its IPC connections broken.  There's not much to be done in that case, but the logging could suggest that the other process (I think we have its pid somewhere usable) may have crashed; if that's seen in connection with messages like “application terminated with exit code 11” in a test, or if it's interactive and the application has obviously crashed, the obvious inference is obvious.  I think we can have communication directly between child processes, like a content process and a plugin, so that case needs to be accounted for.

Comment 2

[Intermittent Failures Robot]

3 failures in 788 pushes (0.004 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 3

Platform breakdown:
* linux64-stylo-disabled: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1428509&startday=2018-01-08&endday=2018-01-14&tree=all

Comment 3

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

We do have a way to check for expected shutdown — see bug 529005 — but it's handled in MessageChannel, which is a layer we added on top of the Chromium-derived IPC channel code that's printing this error.  So it looks possible to get the "pipe error" message in normal operation, and also possible to fix that.

We shouldn't need the goodbye-message handling in the intra-process case, so in principle it should be possible to move it down a level and suppress these messages, but that would be a nontrivial change.  And that doesn't help with the part of this where the process does exit unexpectedly.