525605 - E10s: windows unit tests orange, crash under nsNPAPIPlugin::CreatePlugin (IPC off)

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, defect)

Description

[Benjamin Smedberg]

E10s windows unit tests are orange:

Last output:
6950 INFO Running chrome://mochikit/content/chrome/modules/plugin/test/test_bug479979.xul...


Crash reason:  EXCEPTION_ACCESS_VIOLATION
Crash address: 0x0

Thread 0 (crashed)
 0  0x4f0fe92
    eip = 0x04f0fe92   esp = 0x0012e5f8   ebp = 0x1056df4a   ebx = 0x03855630
    esi = 0x0012e604   edi = 0x04f0fe90   eax = 0x00000000   ecx = 0x0012e600
    edx = 0x00618640   efl = 0x00010246
    Found by: given as instruction pointer in context
 1  xul.dll!nsNPAPIPlugin::CreatePlugin(char const *,PRLibrary *,nsIPlugin * *) [nsNPAPIPlugin.cpp:0f0ff5b8d5d0 : 461 + 0x20]
    eip = 0x105703db   esp = 0x0012e60c   ebp = 0x1056df4a
    Found by: stack scanning
 2  xul.dll!CreateNPAPIPlugin [nsPluginHost.cpp:0f0ff5b8d5d0 : 3963 + 0xd]
    eip = 0x105657a5   esp = 0x0012e640   ebp = 0x1056df4a
    Found by: stack scanning

`make mochitest` works for me locally, I think, but I haven't spent a lot of time at it. In these tests the IPC pref is off, so we should only be using the in-process codepaths.

Comment 1

[Benjamin Smedberg]

Oh, other suites also crash, such as mochitest-browser-chrome:
TEST-PASS | chrome://mochikit/content/browser/browser/base/content/test/browser_pluginnotification.js | Should have a test plugin
and then a crash

Comment 2

[(not currently active) Ted Mielczarek]

That test uses the plugin host to find the test plugin, so something is definitely wrong if it can't find it:
http://mxr.mozilla.org/mozilla-central/source/browser/base/content/test/browser_pluginnotification.js#9

Comment 3

[Benjamin Smedberg]

More information: I can reproduce this with packaged unit tests. We crash in nsNPAPIPlugin::nsNPAPIPlugin on this line:

  nsresult gepResult = aLibrary->NP_GetEntryPoints(&fCallbacks, &gepError);

The function appears to run correctly but when it returns something on the stack is corrupted and it returns to some unexpected location.

Comment 4

[Benjamin Smedberg]

http://hg.mozilla.org/projects/electrolysis/rev/79f4a6b45881
http://hg.mozilla.org/projects/electrolysis/rev/72bbf8383143

The calling convention for the imported functions NP_GetEntryPoints, NP_Initialize and NP_Shutdown was wrong: these functions use WINAPI (stdcall) instead of cdecl. I'm going to leave this bug open until the tbox goes green, but I have confidence!

Comment 5

[Josh Aas]

Sorry about that! Thanks for the fix.

Comment 6

[Benjamin Smedberg]

Yeah, that fixed the majority of the orange. I'll file a separate bug for mochitest-ipcplugins failing due to leaks (I'm going to disable the NPN_SetException test for now, since we don't really care).