1438391 - SysV IPC used by VirtualGL libvglfaker.so preload library

Status: RESOLVED FIXED

(Core :: Security: Process Sandboxing, defect, P1)

Description

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

VirtualGL (https://virtualgl.org/About/Introduction; not to be confused with the Virgil GPU virtualization project) is using a preload library to, apparently, redirect 3D rendering to a different X server from 2D rendering.

It seems to use a preload library, and it creates SysV shared memory early, possibly in an interception of XOpenDisplay or maybe at static initializer time; either way, the syscall we're seeing shmdt (i.e., unmapping) and there's a SysV shared memory mapping in the crash report's module list (and also in the call stack, thanks to Breakpad's heuristic stack scanning).

This should be detectable by checking for the library with RTLD_NOLOAD.

Comment 1

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

I took a look at the code.  The uses of SysV shm appear to be guarded by XShmQueryExtension, so in theory they shouldn't be getting called.  Maybe someone has patches that change it to use XCB, like what I suspect is going on in bug 1438401?

Also, it's not as easy as I'd hoped to detect the library directly — we'd have to do that in glxtest() and then signal it back to the main process somehow.

But the vglrun script sets the VGL_ISACTIVE environment variable, so that ought to work.

And it occurs to me that if we've going to allow SysV IPC then we don't need to suppress XShmQueryExtension.

Comment 2

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

So it's worse than that.  There's a shmget() to set up a segment, for each process, that the vglconfig command can attach to to read and change the process's config struct.  That's not guarded by XShmQueryExtension (it's not being used for X11 MIT-SHM), and it's probably the subject of the shmdt() calls we see.

But also, it seems to lazily connect to the 3D X server when GL is initialized, which is more of a problem.  I'm going to handle this by clamping the effective sandbox level at 3 (i.e., allow network/sockets) if VirtualGL in use, same as for non-remoted PulseAudio.

Comment 3

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

(In reply to Jed Davis [:jld] (⏰UTC-7) from comment #1)
> And it occurs to me that if we've going to allow SysV IPC then we don't need
> to suppress XShmQueryExtension.

Also, this is a bad idea.  libvglfaker.so depends on libXext.so and is preloaded, which means it's an instance of what I speculated in bug 1376910 comment #14: it undoes the fix for bug 1271100, and the shim in libmozsandbox will, as a side effect, re-fix it.

(Most of this stops being a problem if WebGL is remoted to a GPU process, but the X11 connections would probably still be an issue.  I notice that VirtualGL's docs include a list of per-application advice, and the item for Chrome/Chromium says to pass it --disable-gpu-sandbox.)

Comment 4

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

(In reply to Jed Davis [:jld] (⏰UTC-7) from comment #3)
> Also, this is a bad idea.  libvglfaker.so depends on libXext.so and is
> preloaded, which means it's an instance of what I speculated in bug 1376910
> comment #14: it undoes the fix for bug 1271100, and the shim in
> libmozsandbox will, as a side effect, re-fix it.

…except that bug 1271100 is about Cairo using XShm in the parent process, not the content process, and libmozsandbox doesn't affect that, so ignore this.  In any case, that shim is probably going away in bug 1438401 now that I've had a better chance to look at it.

Comment 5

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Attachment: Bug 1438391 - Detect VirtualGL and weaken the sandbox enough for it to work.

Review commit: https://reviewboard.mozilla.org/r/223736/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/223736/

Comment 6

[Gian-Carlo Pascutto [:gcp]]

Comment on attachment 8954650 [details]
Bug 1438391 - Detect VirtualGL and weaken the sandbox enough for it to work.

https://reviewboard.mozilla.org/r/223736/#review229802

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fca779af7ef7
Detect VirtualGL and weaken the sandbox enough for it to work. r=gcp

Comment 8

[Tiberius Oros[:tiberius_oros]]

https://hg.mozilla.org/mozilla-central/rev/fca779af7ef7