1401774 - The Linux Sandbox file broker should handle fd exhaustion more gracefully

Status: RESOLVED FIXED

(Core :: Security: Process Sandboxing, enhancement, P3)

Description

[Jed Davis [:jld] ⟨⏰|UTC-8⟩ ⟦he/him⟧]

When sending file descriptors with SCM_RIGHTS, if the receiving process can't allocate a descriptor — usually because it's reached the RLIMIT_NOFILE limit — the kernel truncates the control message at that point, removes it completely if it would be empty, and sets the MSG_CTRUNC flag in msg_flags.  We interpret this as hostile input and hang up the connection, causing all following requests to fail.

But we can detect this case.  In the special case where there's exactly one fd and no other control messages, which covers all of the broker's messages, this is ((msg_flags & MSG_CTRUNC) != 0 && msg.msg_controllen == 0).

If this happens in the parent process — which is probably the more likely case — it affects the response socket, so the client will get EOF and there's nothing the parent can do about it, but we can at least print the error message for EMFILE instead of EMSGSIZE.

If this affects the child process for the response to an open() call, it can return EMFILE.  The client could turn an EOF into EMFILE, but at best that's weird because it's a different process's file limit, and at worst it's a completely different reason like the parent process exiting, so that seems like a bad idea.

(While we're here, we might want to consider turning EPIPE or other socket errors into EIO.  On the one hand, EPIPE isn't a normal response to open()/access()/stat()/etc. and that might possibly confuse the caller; on the other hand, seeing “broken pipe” in an error message is a pretty clear sign that it's the broker.)

I'm also not sure if we should just return an error and continue on, or if maybe there should be a MOZ_ASSERT(false) in there to make the fd exhaustion more obvious, at least on debug builds (and if it happens to be the broker that runs into it, rather than IPC or something else).

Comment 1

[Jed Davis [:jld] ⟨⏰|UTC-8⟩ ⟦he/him⟧]

Attachment: Bug 1401774 - Make fd exhaustion errors from the Linux sandbox broker more obvious.

If we get MSG_CTRUNC back from recvmsg in this context, it means
a file descriptor couldn't be received; because the sender will never
attach too many fds, the only reasonable cause is fd exhaustion in the
receiving process. Therefore, we should return EMFILE ("Too many open
files") instead of EMSGSIZE ("Message too long") to reduce confusion
when reading log messages.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/babff694ebe5
Make fd exhaustion errors from the Linux sandbox broker more obvious. r=gcp

Comment 3

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/babff694ebe5