555699 - OOPP: Java's Signature Warning window is not detected by hang detector [@ hang | ntdll.dll@0x1f861 ]

Status: RESOLVED WORKSFORME

(Core Graveyard :: Plug-ins, defect)

Description

[Andreas Jung]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a4pre) Gecko/20100328 Minefield/3.7a4pre (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a4pre) Gecko/20100328 Minefield/3.7a4pre

If you open the provided URL the Java Plug-in will prompt you with a signature warning. If you ignore this warning by switching to another application etc. it will crash the Java Plug-in.

If you choose send crash report it will actually create two crash reports?!

Crash reports:
http://crash-stats.mozilla.com/report/index/f4b7cfff-1254-459b-b085-082402100329
http://crash-stats.mozilla.com/report/index/7de93a9c-cc7f-4730-b2d8-dde552100329

http://crash-stats.mozilla.com/report/index/d45f4bea-b636-4aa1-b489-3f53d2100329
http://crash-stats.mozilla.com/report/index/180064d5-ce14-4fa9-8bdf-2b2482100329

Reproducible: Always

Steps to Reproduce:
1.Open provided URL
2.Switch to another application
3.(wait a few seconds)

Actual Results:  
The Java plug-in crashes.


Expected Results:  
The Java plug-in doesn't crash.


Sun Java
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) Client VM (build 14.3-b01, mixed mode, sharing)

Comment 1

[timeless]

two reports mean that you've deadlocked, so there's one for firefox and one for the plugin host (of java)

firefox:
mozilla::plugins::PluginModuleParent::ShouldContinueFromReplyTimeout	 dom/plugins/PluginModuleParent.cpp:224
mozilla::ipc::SyncChannel::ShouldContinueFromTimeout	ipc/glue/SyncChannel.cpp:258
mozilla::ipc::RPCChannel::Call	ipc/glue/RPCChannel.cpp:214
mozilla::plugins::PPluginInstanceParent::CallNPP_GetValue_NPPVpluginScriptableNPObject	obj-firefox/ipc/ipdl/PPluginInstanceParent.cpp:242
mozilla::plugins::PluginInstanceParent::NPP_GetValue	dom/plugins/PluginInstanceParent.cpp:527
mozilla::plugins::PluginModuleParent::NPP_GetValue	dom/plugins/PluginModuleParent.cpp:504
nsNPAPIPluginInstance::GetValueFromPlugin	modules/plugin/base/src/nsNPAPIPluginInstance.cpp:1407
nsNPAPIPluginInstance::GetJSObject	modules/plugin/base/src/nsNPAPIPluginInstance.cpp:1495
nsHTMLPluginObjElementSH::GetPluginJSObject	dom/base/nsDOMClassInfo.cpp:9624
nsHTMLPluginObjElementSH::SetupProtoChain	dom/base/nsDOMClassInfo.cpp:9359
nsObjectFrame::NotifyContentObjectWrapper	layout/generic/nsObjectFrame.cpp:2370
nsObjectFrame::TryNotifyContentObjectWrapper	layout/generic/nsObjectFrame.cpp:2120
nsObjectFrame::Instantiate	layout/generic/nsObjectFrame.cpp:2093 

plugin:
GetQueuedCompletionStatus	
base::MessagePumpForIO::GetIOItem	ipc/chromium/src/base/message_pump_win.cc:520
base::MessagePumpForIO::WaitForIOCompletion	ipc/chromium/src/base/message_pump_win.cc:491
base::MessagePumpForIO::WaitForWork	ipc/chromium/src/base/message_pump_win.cc:484
base::MessagePumpForIO::DoRunLoop	ipc/chromium/src/base/message_pump_win.cc:469
base::MessagePumpWin::RunWithDispatcher	ipc/chromium/src/base/message_pump_win.cc:52
base::MessagePumpWin::Run	ipc/chromium/src/base/message_pump_win.h:78
MessageLoop::RunInternal	ipc/chromium/src/base/message_loop.cc:216
MessageLoop::RunHandler	ipc/chromium/src/base/message_loop.cc:199

Comment 2

[Benjamin Smedberg]

This is the hang detector kicking in. Apparently the code that we use to detect when the plugin spins an event loop (such as displaying a modal dialog) isn't detecting the Java signature dialog. After a 10 second timeout we collect the stack minidump of both the firefox and plugin process, then terminate the plugin process.

Comment 3

[Arie Paap [:wildmyron]]

dupe of bug 549930?

Comment 4

[Benjamin Smedberg]

No. That bug is about XStandard, and probably has a different root cause.

Comment 6

[Arie Paap [:wildmyron]]

The signature for this has changed and the stack for both the parent and plugin is completely different:

firefox:
Signature	hang | mozilla::plugins::PPluginInstanceParent::CallUpdateWindow()
UUID	213bc143-74ae-4333-a54a-16a812100503

plugin:
Signature	hang | KiFastSystemCallRet
UUID	6c9bd420-3837-4cac-903d-9ea402100503

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a5pre) Gecko/20100502
Minefield/3.7a5pre ( .NET CLR 3.5.30729) ID:20100502040440

Comment 7

[Arie Paap [:wildmyron]]

Sorry, I should mention that the original signature mozilla::plugins::PPluginInstanceParent::CallNPP_GetValue_NPPVpluginScriptableNPObject(mozilla::plugins::PPluginScriptableObjectParent**, short*)
does still persist in crash reports. Both are low volume (9 in the last week)

Comment 8

[Alex]

This might be related,

1.6.0_17 works just, but 1.6.0_18 (and higher) always crashes - warning dialog unresponsive.

Steps to reproduce,
1. Find website that uses Java and Java prompts a security warning
2. Firefox will get stuck for ~20 seconds
3. Then the Java(TM) security dialog (Run, Cancel) will pop up.
4. This dialog will be unresponsive to input.
5. ~8 seconds later, dialog goes away, Firefox reports Java Plug-in Crashed.

Reproducible: Always.

Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a5pre) Gecko/20100506 Minefield/3.7a5pre
HP Mini 311, Windows 7 x86_32

Comment 10

[Benjamin Smedberg]

My current thought here is that the Java plugin ought not to be blocking the host (Firefox) on the signature verification dialog. That ought to be presented as an asynchronous dialog so that the browser can continue to function normally (and the user can switch tabs etc) while the signature dialog is presented.

Comment 11

[Calvin Cheung]

I've tried FF 3.6.3 with JRE 6u18 and couldn't reproduce the problem.

Does it only reproduce with FF 3.7a4pre?

Comment 12

[Josh Aas]

You need to be running the plugin out-of-process, so use either FF 3.6.4 beta or FF 3.7pre.

Comment 13

[aravindm]

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1) Gecko/20100208 MozillaDeveloperPreview/3.7a1

It works fine on 3.7a1 with JRE 6u20. couldn't reproduce the problem

Build broken : Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a2pre) Gecko/20100209 Minefield/3.7a2pre

Firefox crashes if i click the cancel button on the Java Security Dialog box and close the browser. 

Crash reports: 
http://crash-stats.mozilla.com/report/index/bp-df7cf9d9-f57f-4ef3-ad9f-f17ad2100513
http://crash-stats.mozilla.com/report/index/bp-afe6e24e-e763-4c98-a018-686602100513

Comment 14

[Kevin Brosnan]

(In reply to comment #13)
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1) Gecko/20100208
MozillaDeveloperPreview/3.7a1 is not a recent enough build to be testing this bug against.

I still see this in the trunk, Mozilla/5.0 (Windows; U; Windows NT 6.1; WOW64; en-US; rv:1.9.3a5pre) Gecko/20100601 Minefield/3.7a5pre. The hang that was detected on my system is Bug 564298, ID: b2076f61-3e9c-4306-82e9-e329b2100601 Signature: hang | mozilla::plugins::PPluginInstanceParent::CallUpdateWindow()

Comment 15

[aravindm]

Build worked : 
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1) Gecko/20100208 MozillaDeveloperPreview/3.7a1

Build broken : 
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a2pre) Gecko/20100209 Minefield/3.7a2pre

Pushlog : 
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=43e818c28059&tochange=19dbabe331ad

Comment 16

[Benjamin Smedberg]

The cause of this problem is well-understood, we don't need a regression range. I believe that this is a bug in the Java plugin and should be addressed by them, not by Firefox, and have contacted Sun/Oracle engineers about it. I'm going to mark it blocking-final just to make sure we have some resolution.

Comment 17

[Calvin Cheung]

With OOPP Firefox 3.7a5pre, I noticed the java plugin module (npjp2.dll) was loaded by both firefox.exe and plugin-container.exe processes.

If the plugin is running within the plugin-container.exe process, why is the plugin module being loaded by the firefox.exe process?

Which process does the plugin window belong to?

When a modal dialog such as a security dialog is being displayed, the java plugin sets up windows hook to filter out mouse and focus/activation events. It also starts a message pump on the thread which instantiates the plugin. The message pump is using the following event mask:
PM_REMOVE | PM_QS_PAINT | PM_QS_POSTMESSAGE | PM_QS_SENDMESSAGE

When the dialog is up, I wasn't seeing the browser tab window being repainted when the dialog was moved on top of the browser window. So we maybe pumping events in the wrong thread. 

I'm not sure what java plugin should do to fix this problem currently and open to suggestions. Many customers expect the security dialog to be modal. Displaying the dialog in a different thread and making it non-modal isn't a good solution.

Comment 18

[Benjamin Smedberg]

The plugin DLL is being loaded in the Firefox process so that we can extract the MIME type and description information on it. We don't run plugin instances in the Firefox process.

The plugin window belongs to the plugin-runtime process.

When a plugin (Java/Flash) presents a native modal dialog, we detect this using a windows hook and spin the Firefox event loop while the modal dialog is displayed. Is it possible that you have installed your windows hook incorrectly and it is not calling ::CallNextHookEx at the end of its function. That would prevent it from chaining to our hook.

Comment 19

[Calvin Cheung]

Our windows hook calls the CallNextHookEx before returning.
We're using SetWindowsHookEx to setup the windows hook such as:

    SetWindowsHookEx(WH_MOUSE, &mouseHookProc, 0, thread);

For the message pump, we're passing in NULL as the window handle. So it should be retrieving messages for all windows associated with the current thread.

As for the painting problem mentioned in my previous comment #17, I noticed that the parent window of the plugin window belongs to a different process. Chorme is also running plugin in a separate process and I'm not seeing the painting issue or this issue (plugin process got terminated).

How does the FF hang detector determines the plugin process is hung?

When is FF 3.7 going to be released?

Comment 20

[Benjamin Smedberg]

The plugin is hung when a NPAPI call doesn't return for 10 seconds and our hook hasn't detected that you've spun a nested modal event loop. Firefox 4 is scheduled to go to beta in 2 weeks and final in October.

Comment 21

[Jim Mathies [:jimm]]

I think the problem here is that the security dialog isn't a dialog, it's a popup window. We set a message filter hook for dialogs, which wouldn't pick up the display of this window. A little testing will confirm this. We might have to switch our filter hook out with a more generic message hook.

Comment 22

[alex_mayorga]

Is bug 577808 a duplicate of this one?

Comment 23

[Silviu C]

Also reproducible with Firefox 4 beta 2

Comment 26

[Jim Mathies [:jimm]]

Is there some way to reset this dialog? I clicked cancel, and now I can't get it to show again.

Comment 27

[Jim Mathies [:jimm]]

oop, ok, now it's back.

Comment 28

[Scoobidiver (away)]

Now, the signatures are not the same :

Signature	hang | ntdll.dll@0x1f861
UUID	361234da-01b1-4942-8295-2e5e22100902

Signature	hang | mozilla::plugins::PPluginInstanceParent::CallNPP_GetValue_NPPVpluginScriptableNPObject(mozilla::plugins::PPluginScriptableObjectParent**, short*)
UUID	37982283-2301-4dcf-98d5-291f22100902

Comment 30

[[not reading bugmail]]

If they are different, could mean they are different bugs.

Comment 31

[Benjamin Smedberg]

The hang can happen at any number of different call sites; the signature isn't going to be deterministic.

Comment 32

[Scoobidiver (away)]

The bug is in the java plugin (see comment 16).
The java plugin signature has not changed :
3  	npjp2.dll  	npjp2.dll@0x149e

Only the propagation of java plugin crash can be different :
+ hang | ntdll.dll@0x1f861 :
      * Now,, with the URL of this bug
      * bug 577808 which is a dupe
      * Now, with the URL of bug 588326
      * bug 591328 which is a dupe
+ hang | KiFastSystemCallRet :
      * this bug
      * bug 588326 which is a dupe

May be [@ hang | KiFastSystemCallRet ] and [@ npjp2.dll@0x149e ] must be added to the bug title ?

Comment 33

[Benjamin Smedberg]

No, please don't. KiFastSystemCallRet is a generic signature that could just about any bug, and we really don't need more information.

Comment 34

[Jim Mathies [:jimm]]

Good loss of dialog interaction test case:

https://as.photoprintit.de/web/84004066/startClient.do?client=java&type=print

Comment 36

[Benjamin Smedberg]

Jim thinks this needs beta coverage.

Comment 37

[Jim Mathies [:jimm]]

(In reply to comment #17)
> With OOPP Firefox 3.7a5pre, I noticed the java plugin module (npjp2.dll) was
> loaded by both firefox.exe and plugin-container.exe processes.
> 
> If the plugin is running within the plugin-container.exe process, why is the
> plugin module being loaded by the firefox.exe process?
> 
> Which process does the plugin window belong to?
> 
> When a modal dialog such as a security dialog is being displayed, the java
> plugin sets up windows hook to filter out mouse and focus/activation events. It
> also starts a message pump on the thread which instantiates the plugin. The
> message pump is using the following event mask:
> PM_REMOVE | PM_QS_PAINT | PM_QS_POSTMESSAGE | PM_QS_SENDMESSAGE
> 
> When the dialog is up, I wasn't seeing the browser tab window being repainted
> when the dialog was moved on top of the browser window. So we maybe pumping
> events in the wrong thread. 
> 
> I'm not sure what java plugin should do to fix this problem currently and open
> to suggestions. Many customers expect the security dialog to be modal.
> Displaying the dialog in a different thread and making it non-modal isn't a
> good solution.

Calvin, is WM_CREATE one of the events your hook filters? (I believe it is based on the fact that our hook does not receive it for your dialog.) But thought I'd ask for confirmation.

This makes it very hard for us to detect these window.

Comment 38

[Jim Mathies [:jimm]]

Actually, we don't receive *any* events in our hook for this window. The only events we get are for our plugin window.

Without something to trigger off of I'm going to have a hard time fixing this.

Comment 39

[Jim Mathies [:jimm]]

So it appears that the security warning is being created by a 3rd process (java.exe) with it's parent set to the browser. The deadlock happens wen the parent sends messages to the child. The child thread is deadlocked waiting for the 3rd java process to return control. (which presumably is waiting on the parent browser to respond to windowing events.)

Comment 40

[Jim Mathies [:jimm]]

(In reply to comment #39)
> So it appears that the security warning is being created by a 3rd process
> (java.exe) with it's parent set to the browser. The deadlock happens wen the
> parent sends messages to the child. The child thread is deadlocked waiting for
> the 3rd java process to return control. (which presumably is waiting on the
> parent browser to respond to windowing events.)

To follow up, so the reason why we lock up is due to the fact that the ui is created in the 3rd process. Hence we are unable to detect it in the child plugin process, so we don't spin our spin loop in parent. So all of our measures to prevent this don't kick in.

Comment 42

[Jim Mathies [:jimm]]

Attachment: move quirks to module v.1

Comment 43

[Jim Mathies [:jimm]]

Attachment: move expose quirk to quirks v.1

Comment 44

[Jim Mathies [:jimm]]

Comment on attachment 483517 [details] [diff] [review]
move quirks to module v.1

not critical work, just some good cleanup on quirks mode.

Comment 46

[Jim Mathies [:jimm]]

Comment on attachment 483517 [details] [diff] [review]
move quirks to module v.1

these landed in another bug.

Comment 47

[Black_Ps`]

status1.9.2: 	unaffected ,it has the same problem, see http://crash-stats.mozilla.com/report/pending/00a688df-1304-4953-8e96-6946b2101230 ,this is not the first crash report i`m sending..

Comment 48

[Black_Ps`]

Ah i think it's different from AdBlock Plus bug..

Comment 49

[Jim Mathies [:jimm]]

Fixed by bug 603417, but technically still a bug in oopp, so I'll leave this open.

Comment 50

[Andreas Jung]

Afaict since bug 874167 was fixed the java plugin is run out-of-process again.

Testing with Firefox 44.0 and Java 8 U71 and different java applets that cause signature warning dialogs (like the one linked in the bug report URL field) or cause a warning because they request unsandboxed execution (like the check java version applet on java.com) I didn't get any hangs, crashes or other problems, therefore I'm resolving this bug as worksforme.