1274282 - Crash in libunwind.dylib@0x3a81 [nsChildView.mm:2250]

Status: RESOLVED FIXED

(Core :: Widget: Cocoa, defect)

Description

[:Paolo Amadini]

This bug was filed from the Socorro interface and is 
report bp-fb777c64-adf9-49d7-8180-eeb402160519.
=============================================================

Comment 1

[:Paolo Amadini]

* thread #1: tid = 0x65bc4, 0x00007fff8c861a81 libunwind.dylib`libunwind::CompactUnwinder_x86_64<libunwind::LocalAddressSpace>::stepWithCompactEncodingFrameless(unsigned int, unsigned long long, libunwind::LocalAddressSpace&, libunwind::Registers_x86_64&, bool) + 565, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x00007fff8c861a81 libunwind.dylib`libunwind::CompactUnwinder_x86_64<libunwind::LocalAddressSpace>::stepWithCompactEncodingFrameless(unsigned int, unsigned long long, libunwind::LocalAddressSpace&, libunwind::Registers_x86_64&, bool) + 565
    frame #1: 0x00007fff8c85f33b libunwind.dylib`libunwind::CompactUnwinder_x86_64<libunwind::LocalAddressSpace>::stepWithCompactEncoding(unsigned int, unsigned long long, libunwind::LocalAddressSpace&, libunwind::Registers_x86_64&) + 41
    frame #2: 0x00007fff8c85f2e1 libunwind.dylib`libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_x86_64>::step() + 113
    frame #3: 0x00007fff886e4adb libobjc.A.dylib`objc_addExceptionHandler + 719
    frame #4: 0x00007fff85794c5a CoreFoundation`_CFDoExceptionOperation + 154
    frame #5: 0x00007fff89430610 AppKit`_NSAppKitThemeLock + 71
    frame #6: 0x00007fff894303af AppKit`GetThemeImage + 89
    frame #7: 0x00007fff894302e2 AppKit`+[NSColor _controlColor] + 43
    frame #8: 0x00007fff893fb40e AppKit`-[NSDynamicSystemColor recacheColor] + 196
    frame #9: 0x00007fff893fb240 AppKit`-[NSDynamicSystemColor initWithSelector:] + 89
    frame #10: 0x00007fff894301ee AppKit`+[NSColor controlColor] + 80
    frame #11: 0x00007fff8953d133 AppKit`-[NSButtonCell(NSButtonCellPrivate) _fillBackground:withAlternateColor:] + 51
    frame #12: 0x00007fff89528cd7 AppKit`-[NSButtonCell drawInteriorWithFrame:inView:] + 1141
    frame #13: 0x00007fff895287c2 AppKit`-[NSButtonCell drawWithFrame:inView:] + 428
    frame #14: 0x000000010310c16e XUL`nsChildView::UpdateTitlebarCGContext(this=0x00000001170d5000) + 3598 at nsChildView.mm:2250
    frame #15: 0x000000010310b325 XUL`nsChildView::PrepareWindowEffects(this=0x00000001170d5000) + 357 at nsChildView.mm:1961
    frame #16: 0x0000000101c24f48 XUL`mozilla::layers::ClientLayerManager::EndTransaction(this=0x0000000127e43ca0, aCallback=0x000000010334fd60, aCallbackData=0x00007fff5fbfba98, aFlags=END_DEFAULT)(mozilla::layers::PaintedLayer*, gfxContext*, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::DrawRegionClip, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags) + 40 at ClientLayerManager.cpp:341
    frame #17: 0x00000001033a8bab XUL`nsDisplayList::PaintRoot(this=<unavailable>, aBuilder=0x00007fff5fbfba98, aCtx=<unavailable>, aFlags=<unavailable>) + 3355 at nsDisplayList.cpp:1889
    frame #18: 0x00000001033de2b8 XUL`nsLayoutUtils::PaintFrame(aRenderingContext=0x0000000000000000, aFrame=0x0000000116e29e90, aDirtyRegion=<unavailable>, aBackstop=<unavailable>, aBuilderMode=<unavailable>, aFlags=<unavailable>) + 5384 at nsLayoutUtils.cpp:3585
    frame #19: 0x000000010340449e XUL`PresShell::Paint(this=0x00000001176adc00, aViewToPaint=<unavailable>, aDirtyRegion=0x00007fff5fbfc430, aFlags=1) + 1742 at nsPresShell.cpp:6608
    frame #20: 0x00000001030d0c48 XUL`nsViewManager::ProcessPendingUpdatesPaint(this=0x0000000116e23ac0, aWidget=0x00000001170d5000) + 520 at nsViewManager.cpp:482
    frame #21: 0x00000001030d0830 XUL`nsViewManager::ProcessPendingUpdatesForView(this=0x0000000116e23ac0, aView=<unavailable>, aFlushDirtyRegion=<unavailable>) + 336 at nsViewManager.cpp:413
    frame #22: 0x00000001030d15b1 XUL`nsViewManager::ProcessPendingUpdates(this=<unavailable>) + 225 at nsViewManager.cpp:1117
    frame #23: 0x00000001030d1472 XUL`nsViewManager::WillPaintWindow(this=0x0000000116e23ac0, aWidget=0x00000001170d5000) + 98 at nsViewManager.cpp:717
    frame #24: 0x00000001030cf7bd XUL`nsView::WillPaintWindow(this=<unavailable>, aWidget=0x00000001170d5000) + 45 at nsView.cpp:1041
    frame #25: 0x00000001031156de XUL`-[ChildView viewWillDraw] [inlined] nsChildView::WillPaintWindow(this=<unavailable>) + 77 at nsChildView.mm:1510
    frame #26: 0x0000000103115691 XUL`-[ChildView viewWillDraw](self=0x00000001171edcf0, _cmd=<unavailable>) + 401 at nsChildView.mm:4039
    frame #27: 0x00007fff89510f31 AppKit`-[NSView viewWillDraw] + 1126
    frame #28: 0x00007fff89510f31 AppKit`-[NSView viewWillDraw] + 1126
    frame #29: 0x00007fff8950fb7a AppKit`-[NSView _sendViewWillDrawInRect:clipRootView:] + 1423
    frame #30: 0x00007fff894f2047 AppKit`-[NSView displayIfNeeded] + 1021
    frame #31: 0x00007fff8950beb6 AppKit`-[NSWindow _reallyDoOrderWindow:relativeTo:findKey:forCounter:force:isModal:] + 1972
    frame #32: 0x00007fff89b33877 AppKit`__71-[NSWindow _doOrderWindow:relativeTo:findKey:forCounter:force:isModal:]_block_invoke2684 + 92
    frame #33: 0x00007fff894e51e8 AppKit`NSPerformWithScreenUpdatesDisabled + 65
    frame #34: 0x00007fff8950b518 AppKit`-[NSWindow _doOrderWindow:relativeTo:findKey:forCounter:force:isModal:] + 970
    frame #35: 0x00007fff8950b0e0 AppKit`-[NSWindow orderWindow:relativeTo:] + 162
    frame #36: 0x00007fff894fc686 AppKit`-[NSWindow makeKeyAndOrderFront:] + 51
    frame #37: 0x00000001031514a0 XUL`nsCocoaWindow::Show(this=0x0000000117477800, bState=<unavailable>) + 1488 at nsCocoaWindow.mm:851
    frame #38: 0x00000001037e4210 XUL`nsXULWindow::SetVisibility(this=0x00000001171ed250, aVisibility=<unavailable>) + 128 at nsXULWindow.cpp:878
    frame #39: 0x00000001037e04fe XUL`nsXULWindow::OnChromeLoaded(this=0x00000001171ed250) + 606 at nsXULWindow.cpp:1104
    frame #40: 0x00000001037e025d XUL`nsWebShellWindow::OnStateChange(this=0x00000001171ed250, aProgress=<unavailable>, aRequest=<unavailable>, aStateFlags=<unavailable>, aStatus=<unavailable>) + 621 at nsWebShellWindow.cpp:610
    frame #41: 0x00000001037e090d XUL`non-virtual thunk to nsWebShellWindow::OnStateChange(this=<unavailable>, aProgress=<unavailable>, aRequest=<unavailable>, aStateFlags=<unavailable>, aStatus=<unavailable>) + 13 at nsWebShellWindow.cpp:614
    frame #42: 0x0000000101a61d22 XUL`nsDocLoader::DoFireOnStateChange(this=0x0000000117420800, aProgress=0x0000000117420828, aRequest=0x0000000113d48968, aStateFlags=0x00007fff5fbfd1dc, aStatus=NS_OK) + 306 at nsDocLoader.cpp:1250
    frame #43: 0x0000000101a6190b XUL`nsDocLoader::doStopDocumentLoad(this=<unavailable>, request=0x0000000113d48968, aStatus=NS_OK) + 315 at nsDocLoader.cpp:845
    frame #44: 0x0000000101a60a38 XUL`nsDocLoader::DocLoaderIsEmpty(this=0x0000000117420800, aFlushLayout=<unavailable>) + 616 at nsDocLoader.cpp:724
    frame #45: 0x0000000101a61426 XUL`nsDocLoader::OnStopRequest(this=0x0000000117420800, aRequest=<unavailable>, aCtxt=<unavailable>, aStatus=<unavailable>) + 950 at nsDocLoader.cpp:608
    frame #46: 0x0000000101a6179a XUL`non-virtual thunk to nsDocLoader::OnStopRequest(this=<unavailable>, aRequest=<unavailable>, aCtxt=<unavailable>, aStatus=<unavailable>) + 10 at nsDocLoader.cpp:612
    frame #47: 0x00000001011ce232 XUL`nsLoadGroup::RemoveRequest(this=0x0000000117484600, request=0x0000000112daba80, ctxt=0x0000000000000000, aStatus=NS_OK) + 930 at nsLoadGroup.cpp:633
    frame #48: 0x0000000101eb2046 XUL`nsDocument::DoUnblockOnload(this=0x0000000116e27000) + 246 at nsDocument.cpp:9229
    frame #49: 0x0000000101ee45ca XUL`nsUnblockOnloadEvent::Run(this=<unavailable>) + 10 at nsDocument.cpp:9182
    frame #50: 0x000000010113e178 XUL`nsThread::ProcessNextEvent(this=0x000000010030d1f0, aMayWait=<unavailable>, aResult=0x00007fff5fbfd60f) + 1208 at nsThread.cpp:1073
    frame #51: 0x0000000101166f6e XUL`NS_ProcessPendingEvents(aThread=<unavailable>, aTimeout=10) + 94 at nsThreadUtils.cpp:232
    frame #52: 0x00000001030f268f XUL`nsBaseAppShell::NativeEventCallback(this=0x00000001111b0620) + 111 at nsBaseAppShell.cpp:97
    frame #53: 0x000000010314c9d4 XUL`nsAppShell::ProcessGeckoEvents(aInfo=0x00000001111b0620) + 308 at nsAppShell.mm:387
    frame #54: 0x00007fff857c55b1 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
    frame #55: 0x00007fff857b6c62 CoreFoundation`__CFRunLoopDoSources0 + 242
    frame #56: 0x00007fff857b63ef CoreFoundation`__CFRunLoopRun + 831
    frame #57: 0x00007fff857b5e75 CoreFoundation`CFRunLoopRunSpecific + 309
    frame #58: 0x00007fff9210aa0d HIToolbox`RunCurrentEventLoopInMode + 226
    frame #59: 0x00007fff9210a7b7 HIToolbox`ReceiveNextEventCommon + 479
    frame #60: 0x00007fff9210a5bc HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 65
    frame #61: 0x00007fff893bb24e AppKit`_DPSNextEvent + 1434
    frame #62: 0x00007fff893ba89b AppKit`-[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 122
    frame #63: 0x000000010314c0b2 XUL`-[GeckoNSApplication nextEventMatchingMask:untilDate:inMode:dequeue:](self=0x0000000100338aa0, _cmd=<unavailable>, mask=18446744073709551615, expiration=0x422d63c37f00000d, mode=0x00007fff7576bd00, flag='\x01') + 82 at nsAppShell.mm:121
    frame #64: 0x00007fff893ae99c AppKit`-[NSApplication run] + 553
    frame #65: 0x000000010314cf84 XUL`nsAppShell::Run(this=<unavailable>) + 276 at nsAppShell.mm:661
    frame #66: 0x0000000103a13a84 XUL`nsAppStartup::Run(this=0x00000001111a2d80) + 36 at nsAppStartup.cpp:284
    frame #67: 0x0000000103a64ffb XUL`XREMain::XRE_mainRun(this=<unavailable>) + 4155 at nsAppRunner.cpp:4369
    frame #68: 0x0000000103a654e7 XUL`XREMain::XRE_main(this=0x00007fff5fbff280, argc=<unavailable>, argv=<unavailable>, aAppData=<unavailable>) + 679 at nsAppRunner.cpp:4473
    frame #69: 0x0000000103a65876 XUL`XRE_main(argc=<unavailable>, argv=<unavailable>, aAppData=<unavailable>, aFlags=<unavailable>) + 262 at nsAppRunner.cpp:4581
    frame #70: 0x0000000100002515 firefox`main [inlined] do_main(argc=<unavailable>, argv=<unavailable>, envp=<unavailable>, xreDirectory=0x0000000100313040) + 661 at nsBrowserApp.cpp:242
    frame #71: 0x0000000100002280 firefox`main(argc=<unavailable>, argv=<unavailable>, envp=<unavailable>) + 1280 at nsBrowserApp.cpp:382
    frame #72: 0x0000000100001a44 firefox`start + 52

Comment 2

[:Paolo Amadini]

This seems to happen on a local build only, Nightly builds on the same profile work fine, so mozregression tests are inconclusive. The only lines I've kept in my local ".mozconfig" file are for setting the object directory and the autoclobber option. This crash also doesn't happen in a debug build.

I'm not sure what to try next, except for figuring out a regression range with local builds, but this will take a lot of time even on a relatively fast machine.

Comment 3

[:Paolo Amadini]

Looks like nsChildView.mm:2250 is the first frame in our code.

Comment 4

[:Paolo Amadini]

Based on the above, I've backed out bug 1192053, bug 1263275, bug 1271867, and bug 1261752 part 3 but the crash still happens. I've also built revision a1f793376c1c and the crash is still there. Trying 5ac6c8d1fc28 now, although maybe this is the wrong path and the crash may be linked to external factors...

Comment 5

[:Paolo Amadini]

Revision 5ac6c8d1fc28 works, so I can probably slowly find a regression range.

Comment 6

[:Paolo Amadini]

Looks like as I approach the regression range, the startup crash does not happen all the time. I haven't figured out if this is because of changes made in the profile folder when a build succeeds, or maybe because of a local or OS level cache. It seems that if the previous startup succeeded, new startups tend to succeed as well, but I'm not sure if it happens all the time.

Comment 7

[:Paolo Amadini]

I've worked around this using artifact builds, so for now I should be unblocked on front-end development at least.

Comment 8

[:Paolo Amadini]

This startup crash on Mac OS X 10.9 seems theme related and may be really specific to my machine, it only happened to me as far as I can tell, and only when I build locally, not on Nightly or artifact builds.

What concerns me is that I'm not sure if we run any automated tests on 10.9 at all in continuous integration, I haven't seen any job in treeherder at least. Maybe this crash would happen on other 10.9 machines in specific cases.

If we ignore 10.9 in automation, I might as well upgrade to 10.10 and this may well solve the crash and restore my ability to use full builds locally, though I'd lose the ability to test theme changes on this earlier version.

I thought I'd needinfo a few people who might suggest an investigation route based on the stack trace in comment 1 or general guidance on crash investigations. Just reset the flag if you have no comments.

Comment 9

[Olli Pettay [:smaug][[e-mail]]]

I know nothing about the stuff on the stack trace.
Something to do with painting and native theming.

Comment 10

[Markus Stange [:mstange]]

A similar crash was fixed in bug 790234 by adding ObjC exception guards. And sure enough, it looks like nsChildView::PrepareWindowEffects is missing those guards. I'll add them.

Comment 11

[Markus Stange [:mstange]]

Attachment: MozReview Request: Bug 1274282 - Add exception guards to nsChildView::PrepareWindowEffects(). r?spohl

Review commit: https://reviewboard.mozilla.org/r/54896/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/54896/

Comment 12

[Markus Stange [:mstange]]

Paolo, does this patch fix the crash for you?

Comment 13

[Stephen A Pohl [:spohl]]

Comment on attachment 8755955 [details]
MozReview Request: Bug 1274282 - Add exception guards to nsChildView::PrepareWindowEffects(). r?spohl

https://reviewboard.mozilla.org/r/54896/#review51636

Comment 14

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/8ea3030332a9

Comment 15

[:Paolo Amadini]

(In reply to Markus Stange [:mstange] from comment #12)
> Paolo, does this patch fix the crash for you?

That's great, the crash is totally fixed! Thanks for the quick response!

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/8ea3030332a9

Comment 17

[Markus Stange [:mstange]]

Great!

I'm curious whether there actually is an NSException being fired. Can you check whether it says anything along the lines of "Mozilla has caught an Obj-C exception" when you launch Firefox? There might also be additional information in the "All Messages" section in Console.app. Paolo, can you check?

Comment 18

[:Paolo Amadini]

(In reply to Markus Stange [:mstange] from comment #17)
> I'm curious whether there actually is an NSException being fired. Can you
> check whether it says anything along the lines of "Mozilla has caught an
> Obj-C exception" when you launch Firefox? There might also be additional
> information in the "All Messages" section in Console.app. Paolo, can you
> check?

I see no terminal output at all, maybe it's because this is a Release build, which was the only type where the crash happened anyways. I've checked Console.app and I don't see anything relevant either.

Is there something else I can check?

Comment 19

[Markus Stange [:mstange]]

I think those were all the relevant places. If there's no output, then I don't think any exception is fired, or at least it doesn't propagate into our own code. That's good. Thanks for checking!