527876 - test_windowed_invalidate.html hangs during mochitest-ipcplugins

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, defect)

Description

[Benjamin Smedberg]

test_windowed_invalidate hangs during execution of mochitest-ipcplugins. This seems to have started after bent's deadlock patch landed. If you run the test by itself, it will hang with a red box, with occasional printfs:

[PluginModuleChild] _releasevariantvalue

If I move the mouse around, I'll get these also:
[PluginModuleWARNING: Received nonqueued message 0x84 during a sync IPC message
for window 0x329622 ("MozillaWindowClass"), sending it to DefWindowProc instead
of the normal window procedure.

If you resize the browser window, or attach MSVC to it, the test finishes properly, with the following output:

[PluginModuleParent] mozilla::plugins::PluginInstanceParent::NPP_SetWindow
WARNING: Received nonqueued message 0x215 during a sync IPC message for window 0
x853134 ("MozillaUIWindowClass"), sending it to DefWindowProc instead of the nor
mal window procedure.
[PluginModuleParent] mozilla::plugins::PluginInstanceParent::NPP_SetWindow

Comment 1

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Fixed with revision ec15b649b679.

This was a subtle problem with the way InvalidateRect works, see http://msdn.microsoft.com/en-us/library/dd183426%28VS.85%29.aspx :

"The system does not set the parent's update region when the child's is set. An application cannot generate a WM_PAINT message for the parent window by invalidating the child window. Similarly, an application cannot generate a WM_PAINT message for the child by invalidating a portion of the parent's client area that lies entirely under the child window. In such cases, neither window receives a WM_PAINT message."

The third sentence was our problem. After talking with jimm we decided to invalidate on the child side as well as sending the notification to the browser (as the browser has extra code that runs there). We may want to revisit this once windowless plugins land and we have more experience with invalidation.

Comment 2

[Ben Turner (not reading bugmail, use the needinfo flag!)]

After talking to roc I've pushed changeset d5d054a09eb7 as well. We don't need to send the invalidation message to the parent for windowed plugins.