1780312 - Crash in [@ __statfs64]

Status: RESOLVED FIXED

(Core :: Security: Process Sandboxing, defect, P1)

Description

[Andrew McCreight [:mccr8]]

Crash report: https://crash-stats.mozilla.org/report/index/f9a50afe-1fce-46f5-be44-695de0220718

Reason: SIGSYS / SYS_SECCOMP

Top 10 frames of crashing thread:

0 libc.so.6 __statfs64 /usr/src/debug/glibc/sysdeps/unix/sysv/linux/statfs64.c:34
1 libnvidia-glcore.so.515.57 _nv011glcore 
2 libnvidia-glcore.so.515.57 _nv011glcore 
3 libnvidia-glcore.so.515.57 _nv011glcore 
4 libGLX_nvidia.so.0 <.text ELF section in libGLX_nvidia.so.515.57> 
5 libGLX_nvidia.so.0 vk_icdNegotiateLoaderICDInterfaceVersion 
6 libGLX_nvidia.so.0 <.init ELF section in libGLX_nvidia.so.515.57> 
7 ld-linux-x86-64.so.2 call_init /usr/src/debug/glibc/elf/dl-init.c:26
8 ld-linux-x86-64.so.2 _dl_init /usr/src/debug/glibc/elf/dl-init.c:117
9 libc.so.6 _dl_catch_exception /usr/src/debug/glibc/elf/dl-error-skeleton.c:182

Comment 1

[Andrew McCreight [:mccr8]]

rdd process, MediaPD~oder thread, inside some kind of graphics driver-y looking code?

Comment 2

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

We ran into something similar with Mesa in the RDD process, but if I recall correctly it went away when we turned off the shader cache (which isn't useful when the workload is just media decoding). It's possible that nvidia's version of that, which looks like it'd be __GL_SHADER_DISK_CACHE=0, would work for this.

It's also possible to allow statfs the same way we did in content processes: intercept it and substitute open (which is brokered) + fstatfs (which would need to be allowed).

Comment 3

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Attachment: Bug 1780312 - Turn off the Linux nvidia driver's shader cache in the RDD process.

We were already turning off Mesa's shader cache in the RDD process,
because it's not useful given that we're only using video codec
acceleration and moving images around, and it does a few things related
to trying to access the cache that the sandbox would have to accomodate.

This patch does the equivalent thing for the nvidia proprietary driver;
we don't support it for media codec acceleration, but it can still be
loaded in that process (e.g., on multi-GPU systems) and it's trying to
call statfs on startup which may be related.

Comment 4

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

I don't have a setup that can reproduce this, so I don't know if this patch will prevent the statfs calls, but it seems like a reasonable thing to do given that we're already doing the same thing for Mesa. I'm planning to land this and then check back in a week or two to see if the crash reports have stopped.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1ba53f776c9a
Turn off the Linux nvidia driver's shader cache in the RDD process. r=gcp

Comment 6

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Reminder to myself to check the crash stats in a week or two.

Comment 7

[Cristina Cozmuta (:CrissCozmuta)]

https://hg.mozilla.org/mozilla-central/rev/1ba53f776c9a

Comment 8

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

No crashes in ~10 days, so that probably worked.

I'm marking this wontfix for ESR102 because VA-API isn't enabled there.

And I'm marking this wontfix for beta because this won't crash there, and we don't support nvidia for video codec acceleration so any fallout from (non-fatally) rejecting the syscall probably won't affect anything user-visible.

Comment 9

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Looks like that didn't work: bug 1787714 is basically the same stack, but the crash signature changed (which I think is coincidence). I'll dust off my patch for allowing statfs.

Comment 11

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Attachment: Bug 1780312 - Part 1: Move the statfs replacement into the common sandbox policy.

We have code to handle statfs calls in content processes by
intercepting them and calling open and fstatfs instead; the former
is then recursively intercepted and brokered. This patch moves that
feature into the common policy, but does not allow fstatfs in any
other sandbox types (yet; see next patch). This doesn't affect security
because the caller could have attempted the open and fstatfs
syscalls itself.

Comment 12

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Attachment: Bug 1780312 - Part 2: Allow fstatfs in the Linux RDD sandbox policy.

As discussed in the last patch, allowing fstatfs will also make
statfs work on any path that the process could open for reading
(subject to sandbox policy).

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

The bug is linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

:jld, could you consider increasing the severity of this top-crash bug?

For more information, please visit auto_nag documentation.

Comment 14

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

This won't crash on non-Nightly, and also this depends on a feature that hasn't ridden the trains yet.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/42d3c880806e
Part 1: Move the statfs replacement into the common sandbox policy. r=gcp
https://hg.mozilla.org/integration/autoland/rev/5ff886a06c2f
Part 2: Allow fstatfs in the Linux RDD sandbox policy. r=gcp

Comment 16

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/42d3c880806e
https://hg.mozilla.org/mozilla-central/rev/5ff886a06c2f