Created attachment 503720 [details] testcase Assertion failure: !cx->isExceptionPending(), at js/src/jscntxtinlines.h:695 The stack trace for the assertion is very long, but an opt build does not crash.
Oh, I bet this is a bug involving JS_CHECK_RECURSION recovery.
I think this bug is making it difficult to create reduced testcases for other JS_CHECK_RECURSION bugs, some of which may be more serious.
Created attachment 569174 [details] [diff] [review] Fix XPCConvert error handling I don't know this code either, but the test case and assert seemed nice and straightforward so I took a quick look. The debugger pointed to code that seemed highly suspicious. This patch fixes the attached test case and seems better, at least, than what's there now. I may get lazy and mark this as checkin-needed, though. The problem seems to be that there are at least half a dozen ways for XPCConvert::NativeInterface2JSObject to return false without setting the nsresult to an error value, because it gets whitewashed prematurely.
Comment on attachment 569174 [details] [diff] [review] Fix XPCConvert error handling Yep, my bad. Thanks for catching this.