490929 - Crash [@ DrawCellWithScaling][@ nsNativeThemeCocoa::DrawFrame] on the Mac with large scaled button

Status: RESOLVED WORKSFORME

(Core :: Widget: Cocoa, defect)

Description

[Martijn Wargers (dead)]

Attachment: testcase

See testcase, which crashes current mac trunk build on load.
Perhaps this is related to bug 410415 and/or bug 435223?

http://crash-stats.mozilla.com/report/index/28d8c850-c212-44a3-a693-f7ffb2090430?p=1
0  	CoreUI  	CoreUI@0x316b7  	
1 	CoreUI 	CoreUI@0x1781b 	
2 	CoreUI 	CoreUI@0x1db22 	
3 	AppKit 	AppKit@0x8b32b 	
4 	AppKit 	AppKit@0x8adcf 	
5 	AppKit 	AppKit@0x8ab46 	
6 	XUL 	DrawCellWithScaling 	widget/src/cocoa/nsNativeThemeCocoa.mm:346
7 	XUL 	nsNativeThemeCocoa::DrawPushButton 	widget/src/cocoa/nsNativeThemeCocoa.mm:725
8 	XUL 	nsNativeThemeCocoa::DrawWidgetBackground 	widget/src/cocoa/nsNativeThemeCocoa.mm:1607
9 	XUL 	nsCSSRendering::PaintBackgroundWithSC 	layout/base/nsCSSRendering.cpp:1509
10 	XUL 	nsCSSRendering::PaintBackground 	layout/base/nsCSSRendering.cpp:1342
11 	XUL 	nsButtonFrameRenderer::PaintBorderAndBackground 	layout/forms/nsButtonFrameRenderer.cpp:272
12 	XUL 	nsDisplayButtonBorderBackground::Paint 	layout/forms/nsButtonFrameRenderer.cpp:183
13 	XUL 	nsDisplayList::Paint const 	layout/base/nsDisplayList.cpp:317
14 	XUL 	nsDisplayTransform::Paint 	layout/base/nsDisplayList.cpp:1240
15 	XUL 	nsDisplayList::Paint const 	layout/base/nsDisplayList.cpp:317
16 	XUL 	nsDisplayClip::Paint 	layout/base/nsDisplayList.cpp:1023
17 	XUL 	nsDisplayList::Paint const 	layout/base/nsDisplayList.cpp:317
18 	XUL 	nsLayoutUtils::PaintFrame 	layout/base/nsLayoutUtils.cpp:1104
19 	XUL 	PresShell::Paint 	layout/base/nsPresShell.cpp:5573
20 	XUL 	nsViewManager::RenderViews 	view/src/nsViewManager.cpp:609
21 	XUL 	nsViewManager::Refresh 	view/src/nsViewManager.cpp:511
22 	XUL 	nsViewManager::DispatchEvent 	view/src/nsViewManager.cpp:1105
23 	XUL 	HandleEvent 	view/src/nsView.cpp:167
24 	XUL 	nsChildView::DispatchEvent 	widget/src/cocoa/nsChildView.mm:1956
25 	XUL 	nsChildView::DispatchWindowEvent 	widget/src/cocoa/nsChildView.mm:1969
26 	XUL 	-[ChildView drawRect:] 	widget/src/cocoa/nsChildView.mm:2950
27 	AppKit 	AppKit@0x10929b

Comment 1

[Martijn Wargers (dead)]

Attachment: testcase2

Very likely a similar crash, with this testcase.

Apple crash log:
Thread 0 Crashed:
0   com.apple.coreui              	0x969b66b7 CUIElement::Load(_QSIContext**, CUIContext const*, long, long, long) + 227
1   com.apple.coreui              	0x9699c81c CUIRenderer::Draw9Piece(long, CUIContext const*, unsigned long) + 200
2   com.apple.coreui              	0x969a2b23 CUIRenderer::Draw(CGRect, CGContext*, __CFDictionary const*, __CFDictionary const**) + 2979
3   com.apple.HIToolbox           	0x90b0910e _HIThemeDrawFrameSquare(CGRect const*, HIThemeFrameDrawInfo const*, CGContext*) + 394
4   com.apple.HIToolbox           	0x90a7b169 HIThemeDrawFrame + 159
5   libwidget_mac.dylib           	0x12394a54 nsNativeThemeCocoa::DrawFrame(CGContext*, unsigned long, CGRect const&, int, int) + 670 (nsNativeThemeCocoa.mm:968)
6   libwidget_mac.dylib           	0x1239a6c6 nsNativeThemeCocoa::DrawWidgetBackground(nsIRenderingContext*, nsIFrame*, unsigned char, nsRect const&, nsRect const&) + 5432 (nsNativeThemeCocoa.mm:1713)
7   libgklayout.dylib             	0x12e2e895 nsCSSRendering::PaintBackgroundWithSC(nsPresContext*, nsIRenderingContext&, nsIFrame*, nsRect const&, nsRect const&, nsStyleBackground const&, nsStyleBorder const&, int, nsRect*) + 305 (nsCSSRendering.cpp:1510)
8   libgklayout.dylib             	0x12e2f21d nsCSSRendering::PaintBackground(nsPresContext*, nsIRenderingContext&, nsIFrame*, nsRect const&, nsRect const&, int, nsRect*) + 275 (nsCSSRendering.cpp:1343)
9   libgklayout.dylib             	0x12e4027b nsDisplayBackground::Paint(nsDisplayListBuilder*, nsIRenderingContext*, nsRect const&) + 183 (nsDisplayList.cpp:627)
10  libgklayout.dylib             	0x12e3f917 nsDisplayList::Paint(nsDisplayListBuilder*, nsIRenderingContext*, nsRect const&) const + 61 (nsDisplayList.cpp:316)
11  libgklayout.dylib             	0x12e40c1b nsDisplayWrapList::Paint(nsDisplayListBuilder*, nsIRenderingContext*, nsRect const&) + 41 (nsDisplayList.cpp:821)
12  libgklayout.dylib             	0x12e4255c nsDisplayTransform::Paint(nsDisplayListBuilder*, nsIRenderingContext*, nsRect const&) + 602 (nsDisplayList.cpp:1224)
13  libgklayout.dylib             	0x12e3f917 nsDisplayList::Paint(nsDisplayListBuilder*, nsIRenderingContext*, nsRect const&) const + 61 (nsDisplayList.cpp:316)
14  libgklayout.dylib             	0x12e894dd PresShell::RenderDocument(nsRect const&, unsigned int, unsigned int, gfxContext*) + 1383 (nsPresShell.cpp:5149)
15  libgklayout.dylib             	0x131c966d nsCanvasRenderingContext2D::DrawWindow(nsIDOMWindow*, float, float, float, float, nsAString_internal const&, unsigned int) + 1023 (nsCanvasRenderingContext2D.cpp:3587)
etc..

Comment 2

[Jesse Ruderman]

I can reproduce on trunk, and I get a similar stack trace.

Comment 3

[Steven Michaud [:smichaud] (Retired)]

Sorry, I've been neglecting this.  I'll get back to it as soon as I can.

Comment 4

[Steven Michaud [:smichaud] (Retired)]

Attachment: Gdb trace of testcase1 (on trunk)

Comment 5

[Steven Michaud [:smichaud] (Retired)]

Attachment: Gdb trace of testcase2 (on trunk)

In my experience, testcase2 takes 10-15 seconds to crash.

Comment 6

[Steven Michaud [:smichaud] (Retired)]

By the way, Jesse, what does "[ccbr]" mean?

Comment 7

[Steven Michaud [:smichaud] (Retired)]

These crashes are a royal pain.  They take place deep in system code,
and are clearly caused by Apple bugs:  Apple's code doesn't back out
gracefully when large allocations fail (it might even fail to check
for NULL returns from malloc()).

We previously worked around similar bugs by disabling scaling when the
dimensions of an object to be drawn were too large (bug 444864, bug
458961 and bug 464589).  We could try making those patches more
general, but I'm sorely tempted to just leave this bug unfixed.  The
crashes don't appear to be exploitable (they don't result from
attempting to access deleted objects).

As best I can tell, malloc() failures don't cause the Objective-C
runtime to throw an exception -- otherwise we could try putting
@try/@catch blocks around the system calls where the crashes happen.

Might we be able to catch malloc() failures with C++ exception
handling?  If so, maybe we should just wait until that's supported.

The only other thing I can think of is *very* hackish -- write our own
malloc() and somehow use it to wrap the system malloc() (so that we
can handle malloc() failures ourselves).  I'm not sure this is even
possible.  But if so, it might work like the following instructions
for substituting libgmalloc.dylib for the system's malloc:

http://developer.apple.com/technotes/tn2004/tn2124.html#SECMALLOCDEBUG

Comment 8

[Steven Michaud [:smichaud] (Retired)]

> otherwise we could try putting @try/@catch blocks around the system
> calls where the crashes happen.

I just tried this.  As expected, it didn't catch the malloc failures.

Comment 9

[Steven Michaud [:smichaud] (Retired)]

I've looked further into this.

I've written a debugging patch based on the techniques I developed for
the debugging patch at bug 520312 comment #27.  More about that in my
next comment.

I've also discovered I can't reproduce these crashes on OS X 10.4.11
or 10.6.2.  So apparently Apple introduced some bugs in OS X 10.5.X
that they've now fixed in 10.6.X.

Comment 10

[Steven Michaud [:smichaud] (Retired)]

Attachment: Debugging patch

Here's the debugging patch I mentioned in my previous comment.  It
actually "fixes" this bug on OS X 10.5.X.  But it's pretty hackish --
so I wouldn't want to land it unless this bug's crashes turn out to be
more urgent than they appear to be.

The main purpose of my patch (for now at least) is to show what
Apple's doing wrong to trigger these crashes.  It confirms what I said
in comment #7:

> Apple's code doesn't back out gracefully when large allocations fail
> (it might even fail to check for NULL returns from malloc()).

My patch uses my newly discovered ability to hook non-Objective-C
functions used from dynamically linked frameworks or libraries.  With
this I'm able to hook calls to malloc() and calloc(), and make them
throw Objective-C exceptions when they fail.

To avoid having malloc() and calloc() throw exceptions that would
never be caught, I also used method swizzling to hook an undocumented
method in the QuartzCore framework, and made malloc() and calloc()
only throw when called from that method.  It so happens that this
bug's malloc() and calloc() failures (the ones that lead to its
crashes) always happen under [CIUIBundleBase dataForResolutionData:].

A tryserver build (on the trunk) will follow in a few hours.

Comment 11

[Steven Michaud [:smichaud] (Retired)]

Here's a tryserver build made with my patch from comment #10:
https://build.mozilla.org/tryserver-builds/[e-mail]-bugzilla490929-debug/bugzilla490929-debug-macosx.dmg

Comment 12

[Steven Michaud [:smichaud] (Retired)]

Note to self:

If I ever do land a version of this patch, I'll need to change

+  @try {
+    ++gMallocThrowLevel;
+    retval = [self CIUIBundleBase_dataForResolutionData:resolutionData];
+    --gMallocThrowLevel;
+  } @catch (NSException *exn) {
+    [exn name], [exn reason]);
+    retval = nil;
+  }

to

+  @try {
+    ++gMallocThrowLevel;
+    retval = [self CIUIBundleBase_dataForResolutionData:resolutionData];
+    --gMallocThrowLevel;
+  } @catch (NSException *exn) {
+    --gMallocThrowLevel;
+    [exn name], [exn reason]);
+    retval = nil;
+  }

Comment 13

[André Reinald]

At some point the bug was fixed either by us or by Apple : none of the 2 testcases crashes FF 21 on 10.8.3

Maybe some of you can test on older systems to confirm ?

Comment 14

[Martijn Wargers (dead)]

The first testcase is hanging Firefox on the Mac now.

Comment 15

[Wayne Mery (:wsmwk)]

(In reply to Martijn Wargers [:mwargers] (QA) from comment #14)
> The first testcase is hanging Firefox on the Mac now.

But is that a different bug?

Comment 16

[Martijn Wargers (dead)]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #15)
> (In reply to Martijn Wargers [:mwargers] (QA) from comment #14)
> > The first testcase is hanging Firefox on the Mac now.
> 
> But is that a different bug?

Yes, but that doesn't happen either anymore. Neither of the testcases cause a hang/crash anymore in the latest Nightly build on MacOSX10.11.3. Marking WFM.