Hang and process kill in various performance tests

RESOLVED FIXED in Firefox 39

Status

defect
RESOLVED FIXED
4 years ago
Last year

People

(Reporter: vporof, Assigned: shu)

Tracking

unspecified
Firefox 39
Dependency tree / graph

Firefox Tracking Flags

(firefox39 fixed)

Details

Attachments

(1 attachment)

No description provided.
Seen again for https://treeherder.mozilla.org/logviewer.html#?job_id=50900&repo=gum
Summary: CRASH: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js | application crashed [@ libSystem.B.dylib + 0x3ba6a] → CRASH: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js, browser_perf-details-04.js | application crashed [@ libSystem.B.dylib + 0x3ba6a]
And https://treeherder.mozilla.org/logviewer.html#?job_id=51007&repo=gum
Summary: CRASH: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js, browser_perf-details-04.js | application crashed [@ libSystem.B.dylib + 0x3ba6a] → CRASH: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js, browser_perf-data-massaging-01.js, browser_perf-details-04.js | application crashed [@ libSystem.B.dylib + 0x3ba6a]
These are timeouts, not crashes.
Summary: CRASH: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js, browser_perf-data-massaging-01.js, browser_perf-details-04.js | application crashed [@ libSystem.B.dylib + 0x3ba6a] → Timeouts: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js, browser_perf-data-massaging-01.js, browser_perf-details-04.js
Blocks: perf-tool-tests
No longer blocks: perf-tool-crashes
https://treeherder.mozilla.org/logviewer.html#?job_id=51043&repo=gum
Summary: Timeouts: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js, browser_perf-data-massaging-01.js, browser_perf-details-04.js → Timeouts: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js, browser_perf-data-massaging-01.js, browser_perf-options-enable-memory-01.js, browser_perf-details-04.js
Duplicate of this bug: 1140074
Fixed by bug 1134029
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1134029
Apparently not.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Summary: Timeouts: browser/devtools/performance/test/browser_perf-options-invert-call-tree-01.js, browser_perf-data-massaging-01.js, browser_perf-options-enable-memory-01.js, browser_perf-details-04.js → Hang and process kill in various performance tests
Duplicate of this bug: 1140848
I've managed to trap one beachball so far. Deadlock?

(lldb) bt
* thread #1: tid = 0xa2583, 0x00007fff95e3448a libsystem_kernel.dylib`__semwait_signal + 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x00007fff95e3448a libsystem_kernel.dylib`__semwait_signal + 10
    frame #1: 0x00007fff925beb98 libsystem_pthread.dylib`pthread_join + 445
  * frame #2: 0x0000000105a106b3 XUL`Thread::Join(this=0x000000012084cb80) + 35 at platform-macos.cc:132
    frame #3: 0x0000000105a1691c XUL`SamplerThread::RemoveActiveSampler(sampler=0x0000000134b14380) + 76 at platform-macos.cc:195
    frame #4: 0x0000000105a109a0 XUL`Sampler::Stop(this=0x0000000134b14380) + 128 at platform-macos.cc:333
    frame #5: 0x0000000105a137fb XUL`mozilla_sampler_stop() + 235 at platform.cpp:816
    frame #6: 0x0000000105a0f569 XUL`profiler_stop() + 9 at GeckoProfilerImpl.h:88
    frame #7: 0x0000000105a0f551 XUL`nsProfiler::StopProfiler(this=0x0000000127f0f220) + 17 at nsProfiler.cpp:87
    frame #8: 0x0000000101735c22 XUL`NS_InvokeByIndex(that=0x0000000127f0f220, methodIndex=4, paramCount=0, params=0x00007fff5fbeab10) + 578 at xptcinvoke_x86_64_unix.cpp:174
    frame #9: 0x0000000102592918 XUL`CallMethodHelper::Invoke(this=0x00007fff5fbeaac8) + 88 at XPCWrappedNative.cpp:2113
    frame #10: 0x00000001025834c7 XUL`CallMethodHelper::Call(this=0x00007fff5fbeaac8) + 263 at XPCWrappedNative.cpp:1450
    frame #11: 0x0000000102562634 XUL`XPCWrappedNative::CallMethod(ccx=0x00007fff5fbeaca0, mode=CALL_METHOD) + 260 at XPCWrappedNative.cpp:1417
    frame #12: 0x0000000102564893 XUL`XPC_WN_CallMethod(cx=0x0000000100429c40, argc=0, vp=0x00000001170e3488) + 931 at XPCWrappedNativeJSOps.cpp:1148
    frame #13: 0x0000000106eebebb XUL`js::CallJSNative(cx=0x0000000100429c40, native=0x00000001025644f0, args=0x00007fff5fbeb6d0)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) + 187 at jscntxtinlines.h:235
    frame #14: 0x0000000106e7fa45 XUL`js::Invoke(cx=0x0000000100429c40, args=CallArgs at 0x00007fff5fbeb6d0, construct=NO_CONSTRUCT) + 1269 at Interpreter.cpp:498
    frame #15: 0x0000000106e99fb5 XUL`Interpret(cx=0x0000000100429c40, state=0x00007fff5fbee5b8) + 51749 at Interpreter.cpp:2596
    frame #16: 0x0000000106e8d4a9 XUL`js::RunScript(cx=0x0000000100429c40, state=0x00007fff5fbee5b8) + 585 at Interpreter.cpp:448
    frame #17: 0x0000000106e7fb86 XUL`js::Invoke(cx=0x0000000100429c40, args=CallArgs at 0x00007fff5fbeedb0, construct=NO_CONSTRUCT) + 1590 at Interpreter.cpp:517
    frame #18: 0x0000000106e69464 XUL`js::Invoke(cx=0x0000000100429c40, thisv=0x00007fff5fbef1b0, fval=0x00007fff5fbef1e0, argc=3, argv=0x00007fff5fbef350, rval=JS::MutableHandleValue at 0x00007fff5fbeeeb0) + 900 at Interpreter.cpp:554
    frame #19: 0x00000001071ce6a7 XUL`js::jit::DoCallFallback(cx=0x0000000100429c40, frame=0x00007fff5fbef3e8, stub_=0x00000001191696c0, argc=3, vp=0x00007fff5fbef340, res=JS::MutableHandleValue at 0x00007fff5fbef298) + 1847 at BaselineIC.cpp:9648
    frame #20: 0x000000010078a03b
Yeah, looks like a deadlock on Sampler::sRegisteredThreadsMutex when trying to stop SamplerThread.

RemoveActiveSampler locks sRegisteredThreadsMutex and tries to join the SamplerThread. SamplerThread is waiting on sRegisteredThreadsMutex in Sampler::Run, and can't join.
Not sure why those methods took the lock in the first place.
Assignee: nobody → shu
Status: REOPENED → ASSIGNED
Attachment #8576534 - Flags: review?(mstange)
Comment on attachment 8576534 [details] [diff] [review]
profiler-deadlock.patch

Review of attachment 8576534 [details] [diff] [review]:
-----------------------------------------------------------------

I agree. These locks were added in https://hg.mozilla.org/mozilla-central/rev/f2e44e02f874 , titled "Port multi-thread support to win/mac", but linux doesn't have that lock in that place.
Attachment #8576534 - Flags: review?(mstange) → review+
https://hg.mozilla.org/mozilla-central/rev/91abcf24159b
Status: ASSIGNED → RESOLVED
Closed: 4 years ago4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 39
Product: Firefox → DevTools
You need to log in before you can comment on or make changes to this bug.