1653667 - SIGXCPU in [@ CallProcess]

Status: RESOLVED FIXED

(Core :: Web Audio, defect, P2)

Description

[Tyson Smith [:tsmith]]

Attachment: unreduced.zip

This issue has subverted detection by our tools because when running with e10s we don't get a crash report of any kind (sanitizer logs, minidump, etc). This makes it very difficult to reduce so the attached test case is reduced. This issue also does not seem to reproduce under rr.

Thread 275 "GraphRunner" received signal SIGXCPU, CPU time limit exceeded.

#0  0x00007fffe770800b in empty () at /builds/worker/checkouts/gecko/js/src/ds/LifoAlloc.h:95
#1  operator() () at /builds/worker/checkouts/gecko/js/src/ds/LifoAlloc.cpp:325
#2  0x00007fffe7707b3c in release () at /builds/worker/checkouts/gecko/js/src/ds/LifoAlloc.cpp:332
#3  0x00007fffe6d9dd08 in releaseFrame () at /builds/worker/checkouts/gecko/js/src/vm/Stack.h:800
#4  popInlineFrame () at /builds/worker/checkouts/gecko/js/src/vm/Stack-inl.h:333
#5  popInlineFrame () at /builds/worker/checkouts/gecko/js/src/vm/Stack-inl.h:817
#6  0x00007fffe6d6f01d in Interpret () at /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:2327
#7  0x00007fffe6d48822 in RunScript () at /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:456
#8  0x00007fffe6d7da4e in InternalCallOrConstruct () at /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:611
#9  0x00007fffe6d7fc09 in InternalCall () at /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:639
#10 0x00007fffe6d7fee7 in Call () at /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:656
#11 0x00007fffe6f214d0 in JS_CallFunctionValue () at /builds/worker/checkouts/gecko/js/src/jsapi.cpp:2783
#12 0x00007fffe1d818f1 in Call () at /builds/worker/workspace/obj-build/dist/include/jsapi.h:1496
#13 CallProcess () at /builds/worker/checkouts/gecko/dom/media/webaudio/AudioWorkletNode.cpp:420
#14 0x00007fffe1d83399 in ProcessBlocksOnPorts () at /builds/worker/checkouts/gecko/dom/media/webaudio/AudioWorkletNode.cpp:559
#15 0x00007fffe1d5e054 in ProcessInput () at /builds/worker/checkouts/gecko/dom/media/webaudio/AudioNodeTrack.cpp:536
#16 0x00007fffe1684131 in ProduceDataForTracksBlockByBlock () at /builds/worker/checkouts/gecko/dom/media/MediaTrackGraph.cpp:1116
#17 0x00007fffe1688b98 in Process () at /builds/worker/checkouts/gecko/dom/media/MediaTrackGraph.cpp:1282
#18 0x00007fffe168a4cb in OneIterationImpl () at /builds/worker/checkouts/gecko/dom/media/MediaTrackGraph.cpp:1409
#19 0x00007fffe12b76ae in Run () at /builds/worker/checkouts/gecko/dom/media/GraphRunner.cpp:114
#20 0x00007fffda88404d in ProcessNextEvent () at /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1234
#21 0x00007fffda88ee6d in NS_ProcessNextEvent(nsIThread*, bool) () at /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:513
#22 0x00007fffdbc56755 in Run () at /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:332
#23 0x00007fffdbb32888 in RunInternal () at /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:334
#24 RunHandler () at /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:327

Comment 1

[Tyson Smith [:tsmith]]

Attachment: Detailed Crash Information

Another trace from an ASan debug build m-c 20200717-fa0ad42a026f

Comment 2

[Karl Tomlinson (:karlt)]

SIGXCPU is expected and this can be reported in a debugger even when there is a handler.
Do we know that this signal is terminal?

Usually a signal handler would be installed.
It would not be installed if there is an existing handler, in which case SIGXCPU could incorrectly still be generated. Only the first such thread would be demoted because gRealtimeLimitReached would be reset.
But AFAICS, we don't have any evidence that there is an existing handler involved in this bug.

Comment 3

[Tyson Smith [:tsmith]]

(In reply to Karl Tomlinson (:karlt) from comment #2)

Do we know that this signal is terminal?

Yes that is how we came across it. The process terminates (crash page displayed) but no crash logs are generated (ASan or breakpad).

Comment 4

[Paul Adenot (:padenot)]

We need to understand why the handler and the js interrupt machinery Karl put in doesn't work, then.

Comment 5

[Karl Tomlinson (:karlt)]

Another confusing thing is that we expect the sandbox to prevent atp_promote_current_thread_to_real_time() from succeeding from the content process, but I don't see any prefs set to disable the sandbox. Perhaps the system does not support what the sandbox requires.

Comment 6

[Tyson Smith [:tsmith]]

FWIW we disable sandboxes[1] while fuzzing.

[1] https://github.com/MozillaSecurity/ffpuppet/blob/master/ffpuppet/helpers.py#L367

Comment 7

[Karl Tomlinson (:karlt)]

Attachment: Bug 1653667 move realtime limit initialization from RecvSetProcessSandbox() to CubebUtils r?padenot

so that this happens even when the content sandbox is not enabled.

CubebUtils::InitLibrary() is called during ContentProcess::Init(), before the
event loop is run in XRE_InitChildProcess().

Comment 8

[Karl Tomlinson (:karlt)]

Attachment: Bug 1653667 don't attempt to promote GraphRunner thread on Linux r?padenot

This usually fails due to sandbox policy.

When the sandbox is disabled, the realtime promotion can result in SIGKILL
if/when script runs too long. This can be re-introduced when thread demotion
is moved to a thread that is not blocked on script, which may be part of
the solution to https://bugzilla.mozilla.org/show_bug.cgi?id=1628198.

Depends on D84941

Comment 9

[Karl Tomlinson (:karlt)]

The signal handler was not installed because ContentChild::RecvSetProcessSandbox() is not run when the sandbox is not enabled.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d79ce8620d70
move realtime limit initialization from RecvSetProcessSandbox() to CubebUtils r=padenot
https://hg.mozilla.org/integration/autoland/rev/60e3fc093c79
don't attempt to promote GraphRunner thread on Linux r=padenot

Comment 11

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/d79ce8620d70
https://hg.mozilla.org/mozilla-central/rev/60e3fc093c79

Comment 12

[Release mgmt bot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:karlt, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.

Comment 13

[Karl Tomlinson (:karlt)]

wontfixing other branches because they are only affected when the sandbox is not in effect.