Closed Bug 527122 Opened 10 years ago Closed 10 years ago

"ASSERTION: Failed to initialize nsScriptSecurityManager" and abort/crash, on WINNT leak test build tinderbox

Categories

(Core :: Security, defect)

x86
Windows Server 2003
defect
Not set

Tracking

()

RESOLVED FIXED
mozilla1.9.3a1
Tracking Status
status1.9.2 --- beta3-fixed

People

(Reporter: dholbert, Assigned: vlad)

References

()

Details

(Keywords: assertion, intermittent-failure)

Attachments

(2 files)

{
*** Registering components in: nsBrowserCompsModule
nsNativeModuleLoader::LoadModule("e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\dist\bin\components\i18n.dll") - load FAILED, rv: 80004003, error:
	error 1114
WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80040154: file e:/builds/moz2_slave/mozilla-1.9.2-win32-debug/build/caps/src/nsScriptSecurityManager.cpp, line 3317
###!!! ASSERTION: Failed to initialize nsScriptSecurityManager: 'NS_SUCCEEDED(rv)', file e:/builds/moz2_slave/mozilla-1.9.2-win32-debug/build/caps/src/nsScriptSecurityManager.cpp, line 3395
nsStringStats
 => mAllocCount:          14680
 => mReallocCount:           22
 => mFreeCount:           14119  --  LEAKED 561 !!!
 => mShareCount:          14358
 => mAdoptCount:              8
 => mAdoptFreeCount:          6  --  LEAKED 2 !!!
caps!Construct_nsIScriptSecurityManager+0x000000000000002C (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\caps\src\nssecuritymanagerfactory.cpp, line 362)
xpcom_core!nsGenericFactory::CreateInstance+0x0000000000000026 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\xpcom\build\nsgenericfactory.cpp, line 80)
xpcom_core!nsComponentManagerImpl::CreateInstanceByContractID+0x00000000000000C8 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\xpcom\components\nscomponentmanager.cpp, line 1684)
xpcom_core!nsComponentManagerImpl::GetServiceByContractID+0x00000000000002D5 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\xpcom\components\nscomponentmanager.cpp, line 2251)
xpcom_core!CallGetService+0x000000000000004D (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\xpcom\build\nscomponentmanagerutils.cpp, line 95)
xpcom_core!nsGetServiceByContractID::operator()+0x000000000000001F (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\xpcom\build\nscomponentmanagerutils.cpp, line 278)
xpc3250!nsCOMPtr<nsIScriptSecurityManager>::assign_from_gs_contractid+0x0000000000000019 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\dist\include\nscomptr.h, line 1229)
xpc3250!nsCOMPtr<nsIScriptSecurityManager>::nsCOMPtr<nsIScriptSecurityManager>+0x0000000000000034 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\dist\include\nscomptr.h, line 605)
xpc3250!mozJSComponentLoader::ReallyInit+0x00000000000001A8 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\js\src\xpconnect\loader\mozjscomponentloader.cpp, line 617)
xpc3250!mozJSComponentLoader::LoadModule+0x00000000000000AE (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\js\src\xpconnect\loader\mozjscomponentloader.cpp, line 672)
xpcom_core!nsComponentManagerImpl::AutoRegisterComponent+0x000000000000034D (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\xpcom\components\nscomponentmanager.cpp, line 3040)
xpcom_core!nsComponentManagerImpl::LoadLeftoverComponents+0x000000000000008E (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\xpcom\components\nscomponentmanager.cpp, line 3095)
xpcom_core!nsComponentManagerImpl::AutoRegister+0x000000000000034A (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\xpcom\components\nscomponentmanager.cpp, line 3359)
xpcom_core!NS_InitXPCOM3_P+0x00000000000007D0 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\xpcom\build\nsxpcominit.cpp, line 706)
xul!ScopedXPCOMStartup::Initialize+0x000000000000005B (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\toolkit\xre\nsapprunner.cpp, line 1090)
xul!XRE_main+0x0000000000001BE3 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\toolkit\xre\nsapprunner.cpp, line 3288)
firefox!NS_internal_main+0x00000000000002B2 (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\browser\app\nsbrowserapp.cpp, line 158)
firefox!wmain+0x000000000000011E (e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\toolkit\xre\nswindowswmain.cpp, line 120)
firefox!__tmainCRTStartup+0x00000000000001A6 (f:\sp\vctools\crt_bld\self_x86\crt\src\crtexe.c, line 594)
firefox!wmainCRTStartup+0x000000000000000D (f:\sp\vctools\crt_bld\self_x86\crt\src\crtexe.c, line 414)
kernel32!ProcessIdToSessionId+0x0000000000000209
TEST-UNEXPECTED-FAIL | automation.py | Exited with code 3 during test run
}
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox3.6/1257536447.1257538351.1950.gz
WINNT 5.2 mozilla-1.9.2 leak test build on 2009/11/06 11:40:47
Whiteboard: [orange]
The first instance of this was after my dll blocklisting landed, inthe same commit where component whitelisting went in.  The backout patch still had this issue on L, and it recurred when I relanded component whitelisting... but then went away for a while on backout.

So, this could be related to dll blocklist.  I'm going to land some stuff to add more logging output to the dll blocklisting.
I don't have any idea why LoadModule is being called on i18n.dll at this point: -- it doesn't seem to have actually made it to LdrLoadDll before bailing for whatever reason.  It gets loaded earlier at nsUCvMathModule.  The dll name "usp10.p" is weird; all previous loads look like:

*** Registering components in: nsUCvMathModule
LdrLoadDll: dll name 'i18n.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'gdi32.dll'
LdrLoadDll: continuing load...
*** Registering components in: nsI18nModule
LdrLoadDll: dll name 'necko.dll'
LdrLoadDll: continuing load...
*** Registering components in: necko

Whereas here it seems to have just been skipped entirely, note the lack of nsI18nModule.  The usp10.p is just the display name, we pass down the original args to LdrLoadDll, so I'm not sure why that would break, but something is certainly different here.

...
*** Registering components in: nsUConvModule
LdrLoadDll: dll name 'ucvmath.dll'
LdrLoadDll: continuing load...
*** Registering components in: nsUCvMathModule
LdrLoadDll: dll name 'i18n.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'usp10.p'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'necko.dll'
LdrLoadDll: continuing load...
*** Registering components in: necko
...
*** Registering components in: BrowserDirProvider
LdrLoadDll: dll name 'brwsrcmp.dll'
LdrLoadDll: continuing load...
*** Registering components in: nsBrowserCompsModule
LdrLoadDll: dll name 'mswsock.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'hnetcfg.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'wshtcpip.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'hnetcfg.'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'ws2_32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'iphlpapi.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'iphlpapi'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'rasapi32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'comctl32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'crypt32.pd'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'kernel32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'rtutils.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'shell32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'userenv.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'netapi32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'rasman.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'secur32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'msv1_0.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'shlwapi.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'rtutils.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'netapi32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'rtutils.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'ole32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'xpsp2res.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'ole32'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'ole32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'netshell.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'comctl32.dll'
LdrLoadDll: continuing load...
LdrLoadDll: dll name 'comctl32.dll'
LdrLoadDll: continuing load...
nsNativeModuleLoader::LoadModule("e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\dist\bin\components\i18n.dll") - load FAILED, rv: 80004003, error:
	error 1114
WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80040154: file e:/builds/moz2_slave/mozilla-1.9.2-win32-debug/build/caps/src/nsScriptSecurityManager.cpp, line 3317
###!!! ASSERTION: Failed to initialize nsScriptSecurityManager: 'NS_SUCCEEDED(rv)', file e:/builds/moz2_slave/mozilla-1.9.2-win32-debug/build/caps/src/nsScriptSecurityManager.cpp, line 3395
nsStringStats
Woo!  Finally caught it.  So, here's the weirdness:

LdrLoadDll: Length 22 Buffer 'CLBCatQ.DLL'
LdrLoadDll: fname 'CLBCatQ.DLL'
LdrLoadDll: dll_part 'CLBCatQ.DLL' 11
LdrLoadDll: dll name 'clbcatq.dll'
LdrLoadDll: continuing load...

LdrLoadDll: Length 18 Buffer 'ole32.dll'
LdrLoadDll: fname '.\clbcatq'
LdrLoadDll: dll_part 'clbcatq' 7
LdrLoadDll: dll name 'clbcatq'
LdrLoadDll: continuing load...

we have ole32.dll passed in, but somehow we end up with '.\clbcatq' as the fname.  There is no static data here, so it's not a thread safety or recursion issue here.  The code looks like this:

  int len = moduleFileName->Length / 2;

#ifdef DEBUG
  printf_stderr("LdrLoadDll: Length %d Buffer '%S'\n", moduleFileName->Length, moduleFileName->Buffer);
#endif

  // copy it into fname, which will then be guaranteed null-terminated
  nsAutoArrayPtr<wchar_t> fname = new wchar_t[len+1];
  wcsncpy(fname, moduleFileName->Buffer, len);
  fname[len] = 0; // *ncpy considered harmful

#ifdef DEBUG
  printf_stderr("LdrLoadDll: fname '%S'\n", fname);
#endif

I just don't see any way for fname to end up with anything other than a copy of moduleFileName->Buffer, unless nsAutoArrayPtr is somehow badly screwing up.  This happens in the failed i18n load:

LdrLoadDll: Length 186 Buffer 'e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\dist\bin\components\i18n.dll'
LdrLoadDll: fname 'e:\builds\moz2_slave\mozilla-1.9.2-win32-debug\build\obj-firefox\dist\bin\components\i18n.dll'
LdrLoadDll: dll_part 'i18n.dll' 8
LdrLoadDll: dll name 'i18n.dll'
LdrLoadDll: continuing load...

LdrLoadDll: Length 18 Buffer 'gdi32.dll'
LdrLoadDll: fname '.\usp10.p'
LdrLoadDll: dll_part 'usp10.p' 7
LdrLoadDll: dll name 'usp10.p'
LdrLoadDll: continuing load...

The gdi32.dll load turns into a ".\usp10.p".  Now, when we continue the load, we pass the original buffer down, so the load should still be seeing gdi32.dll... but none of this makes sense to me, without catching this in the compiler.  I've added some more debug code for one more shot at catching this, but I'm about to change this code around to avoid the copy.
Attached patch rollup patchSplinter Review
Here's the current thinking at a potential fix.  Removes all the complex string math, and instead just adds a check to ensure that the string is null terminated, like it always seems to be.  If it's not, we print a warning and continue with the load.

I've already landed this on 1.9.2, in an attempt to fix the crashing leaktest box, but still looking for a review.
Assignee: nobody → vladimir
Attachment #411032 - Flags: review?(shaver)
Comment on attachment 411032 [details] [diff] [review]
rollup patch

>+  // this is for testing -- poke

remove, pls

>+  if (moduleFileName->MaximumLength < moduleFileName->Length+2 ||
>+      fname[len] != 0)
>+  {
>+    printf_stderr("LdrLoadDll: non-null terminated string found!\n");
>+    goto continue_loading;
>+  }

#ifdef DEBUG on the stderr printf? I could go either way for this case, but it looks like you're #ifdef DEBUG everywhere else.

>+#ifdef DEBUG
>+  printf_stderr("LdrLoadDll: dll_part '%S' %d\n", dll_part, len);
>+#endif

This looks like a line of console noise per DLL loaded on all DEBUG builds -- make DEBUG_dll_verbose or some such, please.

>+    if (c >= 'A' && c <= 'Z')
>+      c += 'a' - 'A';
>+

Worth a comment: // Ensure that all ASCII is upper-cased.

>+    if (!load_ok) {
>+      printf_stderr("LdrLoadDll: Blocking load of '%s'\n", dllName);

A link to the equivalent of the blocklist page here in this error message would be spiffy.

r=shaver with nits; how those tests coming? :-)
(In reply to comment #7)
> >+    if (c >= 'A' && c <= 'Z')
> >+      c += 'a' - 'A';
> >+
> 
> Worth a comment: // Ensure that all ASCII is upper-cased.

s/upper/lower/
Attached patch nits patchSplinter Review
About to check in these fixes from review comments, and will also land all this on trunk.  Note that printf_stderr is basically impossible to see in non-debug builds on win32, becuase -console doesn't work on Vista and Win7 any more (it should, see bug 524894), so noone will realistically see the blocklist message, but it's there anyway.
Fixed on both trunk and 1.9.2.  Tests coming soon, probably in the form of a crash test with jsctypes.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Comment on attachment 411526 [details] [diff] [review]
nits patch

>+      printf_stderr("LdrLoadDll: Blocking load of '%s' -- see http://www.mozilla.com/en-US/blocklist/\n", dllName);

Ugh, please don't hardcode Firefox-specific URLs in toolkit code. If you want to include a URL, use the pref service to get the contents of extensions.blocklist.detailsURL and print it.
(In reply to comment #11)
> (From update of attachment 411526 [details] [diff] [review])
> >+      printf_stderr("LdrLoadDll: Blocking load of '%s' -- see http://www.mozilla.com/en-US/blocklist/\n", dllName);
> 
> Ugh, please don't hardcode Firefox-specific URLs in toolkit code. If you want
> to include a URL, use the pref service to get the contents of
> extensions.blocklist.detailsURL and print it.

I've been informed that since this is all pre-xpcom, there's no way to get the pref service. Sigh. Can you at least use http://www.mozilla.com/blocklist/ without the locale?
Argh, it was supposed to be without the locale; I must've had the wrong thing in my cut buffer :(  I'll remove the en-US tomorrow.  Given that you have to change code here to make a change to the blocklist, someone who's doing that can also change the URL if they have a product where they want a different blocklist.
Keywords: assertion
OS: Windows NT → Windows 2000
Target Milestone: --- → mozilla1.9.3a1
OS: Windows 2000 → Windows Server 2003
Whiteboard: [orange]
You need to log in before you can comment on or make changes to this bug.