735090 - "ASSERTION: This is not supposed to fail!: 'Error'" with nearNativeStackLimit

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[Jesse Ruderman]

Attachment: testcase

###!!! ASSERTION: Uh, mDocument doesn't match the current inner window document!: '!GetCurrentInnerWindow() || GetCurrentInnerWindow()->GetExtantDocument() == mDocument', file dom/base/nsGlobalWindow.cpp, line 1888

And during shutdown:

###!!! ASSERTION: scope has non-empty map: '0 == mMainThreadWrappedNativeProtoMap->Count()', file js/xpconnect/src/XPCWrappedNativeScope.cpp, line 287

Also, there's a leak of XPCNativeScriptableInfo, XPCWrappedNativeProto, and nsDOMClassInfo.

Comment 1

[Jesse Ruderman]

Attachment: stack trace

Comment 2

[Jesse Ruderman]

I tested with a 64-bit Mac debug build from Tinderbox. 

https://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-macosx64-debug/1331584352/

To reproduce on another build, you might need to tweak the numeric argument to nearNativeStackLimit, fix bug 735081, or fix bug 735082.

Comment 3

[Daniel Veditz [:dveditz]]

Is the first assertion the same thing as bug 717488? or an unrelated way of triggering that warning?  The shutdown assert doesn't seem like a security issue, more of a leak.

Comment 4

[Jesse Ruderman]

Attachment: testcase 2

Comment 5

[Andrew McCreight [:mccr8]]

The first test case (and the second, with the nearNativeStackLimit changed to 4) now produce this:

###!!! ASSERTION: This is not supposed to fail!: 'Error', file /Users/amccreight/mz/cent3/js/xpconnect/src/nsXPConnect.cpp, line 996
WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file /Users/amccreight/mz/cent3/dom/base/nsGlobalWindow.cpp, line 1686
###!!! ASSERTION: Failed to get script global and holder: 'NS_SUCCEEDED(rv) && newInnerWindow->mJSObject && mInnerWindowHolder', file /Users/amccreight/mz/cent3/dom/base/nsGlobalWindow.cpp, line 1884
WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file /Users/amccreight/mz/cent3/dom/base/nsGlobalWindow.cpp, line 1890
]]]]
++DOMWINDOW == 20 (0x11bc39c80) [serial = 20] [outer = 0x126c45000]
WARNING: window not set for document!: file /Users/amccreight/mz/cent3/layout/base/nsDocumentViewer.cpp, line 1258
Segmentation fault

It looks like what happens is that CreateNativeGlobalForInner fails due to hitting JS_CHECK_RECURSION in JSCompartment::wrap, then somehow we end up in SetNewDocument in the reUseInnerWindow case with currentInner equal to NULL, which this code does not really handle (there is a null check in one place oddly enough) and we get a null deref.

That seems much less scary than the original assertion, and seems more like a DOM bug.  I suppose by tweaking things more it is possible to recreate the original assertion, but I tried a handful of values and wasn't able to.

Comment 6

[Bobby Holley (:bholley)]

Jesse, the dependent bug has now been fixed. Can you still reproduce this?

Comment 7

[Olli Pettay [:smaug][[e-mail]]]

(I couldn't get any relevant assertions when I was testing this earlier today.)

Comment 8

[Bobby Holley (:bholley)]

Ok, I'm going to call this fixed then.