Suspiciously few signatures for macOS content process crashes on Apple Silicon machines
Categories
(Toolkit :: Crash Reporting, defect)
Tracking
()
People
(Reporter: smichaud, Unassigned)
Details
There are many fewer different signatures for macOS content process crashes on ARM64 than there are on AMD64. I suspect this means that Breakpad is either not reporting all the crashes it should, or is reporting them incorrectly (with incorrect signatures).
Here are the macOS content process crashes reported over the last 14 days on Apple Silicon:
1 _pthread_join Add term 1070 79.26 % 1679513 1678226
2 libsystem_pthread.dylib@0x8ad8 Add term 276 20.44 % 1679373
3 IPCError-browser | ShutDownKill | js::NativeGetOwnPropertyDescriptor Add term 1 0.07 %
4 XUL@0x6692280 | XUL@0x248dc48 | XUL@0x248dc48 | XUL@0x248e6cc | XUL@0x258cd1c | XUL@0x73bc20 | XUL@0x685304 | XUL@0x6842f4 | XUL@0x684d3c | XUL@0xee770 | XUL@0xed490 | XUL@0xec730 | XUL@0xf05a4 | XUL@0xfb274 | XUL@0xfecf4 | XUL@0x687368 | XUL@0x648118 ... Add term 1 0.07 %
5 core::ops::function::Fn::call Add term 1 0.07 %
6 mozilla::TrackBuffersManager::UpdateEvictionIndex Add term 1 0.07 %
Here are those reported over the last 14 days on Intel machines:
1 js::GCMarker::processMarkStackTop Add term 4912 8.96 % 1476239 1415708 1400478 1395740 1386875 1257309 952381 941491 792226 772441 745334 730283 719114
2 <name omitted> | std::__1::__sift_up<T> Add term 3686 6.73 % 1654335
3 nsHtml5TreeOperation::Perform Add term 3351 6.12 % 780611 616421
4 js::ObjectGroup::sweep Add term 1911 3.49 % 1582214 1454398 1112741
5 std::__1::__sift_up<T> Add term 1818 3.32 % 1660297
6 _os_lock_corruption_abort | _OSSpinLockLockSlow Add term 1265 2.31 % 1153809
7 js::XDRScript<T> Add term 1128 2.06 % 1444442 790047
8 mozilla::H264::DecodeNALUnit Add term 957 1.75 % 1660368
9 mp4parse_get_indice_table Add term 944 1.72 % 1584956
10 js::gc::TraceEdgeInternal<T> Add term 876 1.60 % 1474623
11 std::__1::__pop_heap<T> Add term 783 1.43 %
12 js::jit::MaybeEnterJit Add term 679 1.24 % 1665579 1665432 1461724 1461480 1451720 1425132 858032
13 core::option::expect_none_failed | lucet_wasi::ctx::WasiCtxBuilder::fd_dup_for_io_desc Add term 676 1.23 % 1670878
14 Interpret Add term 615 1.12 % 1260683 1185961 962141 937914 935348 917792 884194 682573
15 gfxSparseBitSet::SetRange Add term 555 1.01 %
16 js::TenuringTracer::traverse<T> Add term 515 0.94 % 1582214 1358073
17 nsTArray_Impl<T>::AppendElementsInternal<T> | nsHtml5TreeBuilder::startTag Add term 457 0.83 %
18 js::TraceManuallyBarrieredGenericPointerEdge Add term 449 0.82 % 1203273
19 (anonymous namespace)::BytecodeParser::parse Add term 447 0.82 % 1670481
20 GraphWalker<T>::DoWalk Add term 384 0.70 % 1490016 500105
21 mach_port_insert_right Add term 346 0.63 %
22 nsTArray_Impl<T>::ReplaceElementsAtInternal<T> | mozilla::TrackBuffersManager::InsertFrames Add term 307 0.56 %
23 nsCycleCollector::ScanWeakMaps Add term 297 0.54 %
24 nsTArray_Impl<T>::AppendElementsInternal<T> | nsHtml5TreeBuilder::createElement Add term 291 0.53 %
25 mozilla::TrackBuffersManager::GetSample Add term 287 0.52 % 1247189
26 AVF_CreateMediaAcceleratorInterface Add term 282 0.51 %
27 js::jit::ICStub::trace Add term 280 0.51 % 1651704 1247257 1236759 1192988
28 mach_msg_trap Add term 266 0.49 % 1409699 1234370
29 js::GCMarker::eagerlyMarkChildren Add term 261 0.48 % 1441002 1359657 1337578 668583
30 nsGlobalWindowInner::ClearDocumentDependentSlots Add term 259 0.47 % 1576452 1573464 1422313 1405521
31 OOM | unknown | js::AutoEnterOOMUnsafeRegion::crash | pref_Lookup Add term 243 0.44 %
32 js::frontend::GeneralParser<T>::functionFormalParametersAndBody Add term 242 0.44 %
33 nsTArray_Impl<T>::AppendElementsInternal<T> | nsHtml5TreeBuilder::endTag Add term 241 0.44 %
34 vDSP_vclipD Add term 234 0.43 %
35 MD_PrepareForGTPin Add term 231 0.42 %
36 HALObject::Teardown Add term 226 0.41 %
37 CCGraphBuilder::NoteWeakMapping Add term 225 0.41 % 1538552 1125653
38 core::option::expect_none_failed | lucet_wasi::ctx::WasiCtxBuilder::new Add term 221 0.40 %
39 mozilla::dom::JSActorManager::ReceiveRawMessage Add term 209 0.38 % 1563825
40 mozilla::dom::FragmentOrElement::CanSkip Add term 201 0.37 % 791437
41 VideoToolbox@0xff416 Add term 199 0.36 %
42 nsCycleCollector::Collect Add term 197 0.36 % 1528699
43 OSSpinLockLock Add term 196 0.36 %
44 CTFontCopyFamilyName Add term 194 0.35 %
45 js::array_push Add term 192 0.35 %
46 ClearCycleCollectorCleanupData Add term 182 0.33 % 979059
47 js::TenuringTracer::traceObject Add term 179 0.33 % 1494271 1243151
48 IPCError-browser | ShutDownKill | __psynch_cvwait | mozilla::TaskController::GetRunnableForMTTask | mozilla::ipc::MessagePump::Run Add term 176 0.32 % 1674388
49 IOAccelResourceTestEvent Add term 171 0.31 %
50 js::gc::Arena::finalize<T> Add term 167 0.30 % 1358871 702531
Note also the disproportionately large number of the following types of content process crashes on ARM64:
1 _pthread_join Add term 1070 79.26 % 1679513 1678226
2 libsystem_pthread.dylib@0x8ad8 Add term 276 20.44 % 1679373
I suspect that at least some of these crashes actually happened elsewhere (with different signatures).
Comment 1•4 years ago
|
||
There's a race in the crash reporting code that's causing content process crashes to crash the main process. See bug 1644249 and bug 1624467 for the fix.
Reporter | ||
Comment 2•4 years ago
|
||
As best I can tell, this disparity isn't present for browser/main process crashes, or at least not to the same degree:
macOS browser process crashes reported over the last 14 days on Apple Silicon:
1 MacFileUtilities::MachoWalker::FindHeader Add term 1948 74.55 % 1676102
2 CrashReporter::OnChildProcessDumpWritten Add term 502 19.21 % 1644249
3 _pthread_join Add term 74 2.83 % 1679513 1678226
4 libsystem_pthread.dylib@0x8ad8 Add term 33 1.26 % 1679373
5 OUTLINED_FUNCTION_0 Add term 9 0.34 %
6 mozilla::a11y::GeckoTextMarker::ContainerAsHyperTextWrap Add term 8 0.31 % 1679075
7 CrashChannel::OpenContentStream Add term 6 0.23 % 1515987
8 XUL@0x541af3c | XUL@0x541af38 | XUL@0x541d334 | _pthread_start Add term 4 0.15 %
9 shutdownhang | __psynch_cvwait | _pthread_cond_wait Add term 4 0.15 %
10 XUL@0x38a7f38 | XUL@0x38a7f34 | XUL@0x38ab3f8 | XUL@0x38ab1e0 | libsystem_pthread.dylib@0x7068 Add term 3 0.11 %
11 _platform_memcmp | nsAuthGSSAPI::nsAuthGSSAPI Add term 3 0.11 %
12 dyld_stub_binder Add term 3 0.11 % 730633
13 libsystem_pthread.dylib@0x8ad0 Add term 3 0.11 %
14 XUL@0x38b26c8 | XUL@0x38b1e58 | XUL@0x38b1e58 | XUL@0x38b1c9c | XUL@0x38af808 | XUL@0x38af644 | XUL@0x38ae724 | XUL@0x38ad2d0 | XUL@0x38ab360 | XUL@0x38ab1c0 | libsystem_pthread.dylib@0x7068 Add term 2 0.08 %
15 XUL@0x3584cf4 | XUL@0x66f885c | XUL@0x3577804 | XUL@0x1b8468 | XUL@0x1b143c | XUL@0x5ac508 | XUL@0x5a9d54 | XUL@0x5a97d4 | XUL@0x5b52b8 | XUL@0x84561c | XUL@0x757bf4 | XUL@0x6852f8 | XUL@0x6842e8 | XUL@0x684d30 | XUL@0xee770 | XUL@0xed490 | XUL@0xec730... Add term 1 0.04 %
16 XUL@0x388e880 | XUL@0x388e87c | XUL@0x389233c | XUL@0x3892124 | libsystem_pthread.dylib@0x7068 Add term 1 0.04 %
17 XUL@0xa4c4cc | XUL@0x658d570 | XUL@0xa32260 | XUL@0x11176c | XUL@0xa5a72c | XUL@0xa5bd20 | XUL@0x398cea4 | XUL@0x398dfc8 | XUL@0x3b14414 | XUL@0x398fe58 | XUL@0x3986298 | XUL@0x3981634 | XUL@0x398e37c | XUL@0x39b1bf0 | XUL@0x39b1ebc | XUL@0x39b1d50 | X... Add term 1 0.04 %
18 __pthread_kill | pthread_kill Add term 1 0.04 %
19 _dispatch_lane_class_dispose.cold.3 Add term 1 0.04 %
20 _dispatch_main_queue_callback_4CF Add term 1 0.04 %
21 dlopen_internal Add term 1 0.04 %
22 js::ctypes::ConvertToJS Add term 1 0.04 % 1448144 1371207 1363630 718227 630033
23 mozilla::dom::WindowGlobalParent::GetBrowserParent Add term 1 0.04 % 1668682
24 mozilla::dom::ipc::SharedStringMap::SharedStringMap Add term 1 0.04 % 1514734
25 shutdownhang | libsystem_kernel.dylib@0x4548 Add term 1 0.04 %
macOS browser process crashes reported over the last 14 days on Intel:
1 objc_exception_rethrow | -[NSOpenGLLayer drawInOpenGLContext:pixelFormat:forLayerTime:displayTime:] Add term 4414 7.35 % 1601985
2 mozilla::dom::ipc::WritableSharedMap::WritableSharedMap Add term 2431 4.05 % 1490846
3 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | mozilla::detail::MutexImpl::unlock | XPCArrayHomogenizer::GetTypeForArray Add term 2043 3.40 % 1655236
4 __pthread_kill | abort | gpusGenerateCrashLog.cold.1 Add term 1757 2.93 % 1665887 1576767 1535120
5 objc_msgSend | _NSAccessibilityUIElementForSpecifier Add term 1708 2.84 %
6 shutdownhang | __CFRunLoopServiceMachPort Add term 1353 2.25 %
7 NS_CycleCollectorSuspect3 Add term 1085 1.81 % 1586494 1579858 1562232 1544357 1517464 1442854 1402181 1395104 1379913 1377387 1367715 1220667 1198500 1183651 1151643 1133892 1132339 1132199 1116067 1031852 1031697 991626 970767 967985 899761 892588
8 shutdownhang | __psynch_cvwait | _pthread_cond_wait | _pthread_ulock_cond_cleanup Add term 1051 1.75 % 1664472
9 objc_msgSend | CA::Layer::remove_sublayer Add term 790 1.32 % 1658432
10 pthread_mutex_lock | CFRunLoopSourceSignal Add term 728 1.21 % 1664355
11 CrashReporter::OnChildProcessDumpWritten Add term 692 1.15 % 1644249
12 __pthread_kill | abort | gpusGenerateCrashLog Add term 620 1.03 % 1535120
13 AsyncShutdownTimeout | Places Connection shutdown | PlacesUtils read-only connection closing as part of Places shutdown Add term 612 1.02 % 1402103 1326309
14 __pthread_kill | abort | _RegisterApplication Add term 583 0.97 % 1655801
15 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | NS_InvokeByIndex Add term 575 0.96 %
16 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | mozilla::net::nsSocketTransportService::OnDispatchedEvent Add term 556 0.93 %
17 AsyncShutdownTimeout | Places Clients shutdown | sanitize.js: Sanitize on shutdown Add term 508 0.85 % 1578273 1524200 1507171 1426941 1404105
18 js::GCMarker::processMarkStackTop Add term 475 0.79 % 1476239 1415708 1400478 1395740 1386875 1257309 952381 941491 792226 772441 745334 730283 719114
19 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | mozilla::detail::MutexImpl::unlock | nsThread::Shutdown | nsThreadManager::Shutdown Add term 446 0.74 %
20 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | xpc_DumpJSStack Add term 444 0.74 %
21 nsXPCWrappedJS::Release Add term 436 0.73 % 1517464 1427103 1353660 1133892 912110
22 shutdownhang | __psynch_cvwait | _pthread_cond_wait | _pthread_cond_check_init Add term 410 0.68 %
23 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | xpc::AutoScriptActivity::AutoScriptActivity Add term 407 0.68 %
24 MacFileUtilities::MachoWalker::FindHeader Add term 380 0.63 % 1676102
25 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | firefox@0x2ce2 Add term 353 0.59 %
26 _dispatch_client_callout Add term 352 0.59 % 1631803
27 mozilla::dom::ServiceWorkerRegistrar::GetShutdownPhase Add term 313 0.52 % 1603684 1471720 1403348
28 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | mozilla::TaskController::GetRunnableForMTTask | arena_t::MallocSmall | mozilla::detail::MutexImpl::unlock | mozilla::layers::CompositorThreadHolder::Shutdown Add term 302 0.50 %
29 <name omitted> | nsBaseWidget::GetLayerManager Add term 298 0.50 %
30 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | _pthread_cond_updateval Add term 280 0.47 %
31 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | mozilla::ipc::MessageChannel::SynchronouslyClose Add term 273 0.45 %
32 mozilla::XPCOMThreadWrapper::Dispatch Add term 271 0.45 %
33 wr_moz2d_render_cb Add term 261 0.43 % 1664653 1630774 1524280 1516011 1511131 1509870 1486198 1466613 1449545 1449056 1449054 1447997 1447863
34 shutdownhang | libsystem_kernel.dylib@0x1ca16 Add term 256 0.43 % 1676851 1518645
35 __pthread_kill | abort | gpusKillClient Add term 250 0.42 % 1660003 1379408
36 webrender::prepare::prepare_interned_prim_for_render Add term 241 0.40 % 1660675 1511493
37 IOAccelContextAddResource Add term 240 0.40 % 1533143
38 objc_exception_throw | -[NSObject doesNotRecognizeSelector:] | ___forwarding___ | __forwarding_prep_0___ | _OPTCEventHandler Add term 234 0.39 %
39 lookUpImpOrForward Add term 230 0.38 % 1569462
40 alloc::vec::Vec<T>::reserve Add term 229 0.38 %
41 libpkcs11-dnietif.so@0xb2c4 Add term 225 0.37 % 1533422
42 __pthread_kill | abort | cvms_element_get_content_internal Add term 222 0.37 %
43 @0x0 | CamTwist@0x299f Add term 220 0.37 %
44 CFRelease | -[__NSDictionaryM dealloc] Add term 217 0.36 %
45 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | __mac_syscall Add term 216 0.36 %
46 shutdownhang | __psynch_cvwait | _pthread_cond_wait | pthread_cond_signal_thread_np | <name omitted> | WatchdogManager::RecordContextActivity Add term 203 0.34 %
47 _os_unfair_lock_corruption_abort | _os_unfair_lock_lock_slow Add term 201 0.33 %
48 -[ChildView pasteboard:item:provideDataForType:] Add term 196 0.33 %
49 shutdownhang | libsystem_kernel.dylib@0x16db6 Add term 190 0.32 % 1379619 1279691 1268528 1149287
50 objc_msgSend | __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ Add term 186 0.31 %
Reporter | ||
Comment 3•4 years ago
|
||
(In reply to Gabriele Svelto [:gsvelto] from comment #1)
There's a race in the crash reporting code that's causing content process crashes to crash the main process. See bug 1644249 and bug 1624467 for the fix.
Ah yes! OK.
Comment 4•4 years ago
|
||
Crash reporting for AArch64 macs seem to be working now, closing this.
Description
•