1290766 - Crash in Route error: message sent to unknown actor ID msgtype=0xC2000A (PPluginWidget::SetScrollCaptureId)

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, defect)

Description

[Nicholas Nethercote [inactive]]

This bug was filed from the Socorro interface and is 
report bp-ae18165a-be1a-4eac-b301-541512160731.
=============================================================

This crash first appeared in Nightly 20160722030235. I see two variants, one with msgtype=0xC2000A, one with msgtype=0x8E0017. (I'm lumping the two variants into a single bug report because they both started in the same build.) Since then there have been 31+18=49 occurrences across multiple installations.

mrbkap, any ideas? How are those msgtype values constructed? I have trouble determining which msgtype a particular value represents.

Comment 1

[Blake Kaplan (:mrbkap) (inactive)]

(In reply to Nicholas Nethercote [:njn] from comment #0)
> mrbkap, any ideas? How are those msgtype values constructed? I have trouble
> determining which msgtype a particular value represents.

msgtype is constructed by taking the interface (left shifted 16 bits) bitwise or'd with the message number. For the linked report (with msgtype 0xC2000A), we start at <objdir>/ipc/ipdl/_ipdlheaders/IPCMessageStart.h. There, we can find the right IPDL protocol (0xC2 == 194), which looks like it should be PPluginWidgetMsgStart. Now, we can find .../_ipdlheaders/mozilla/plugins/PPluginWidget.h and find the right message (0xA == 10) and we're failing to dispatch a SetScrollCaptureId message.

The only thing I can think of that could be happening here is that we're somehow destroying the child plugin instance between when we create the actors for the plugin and when we receive the SetScrollCaptureId message. That should be very rare though, and I don't know how possible it is.

Comment 2

[Blake Kaplan (:mrbkap) (inactive)]

Oh, and the second message type appears to be PHttpChannel::Redirect3Complete.

Comment 3

[Blake Kaplan (:mrbkap) (inactive)]

Bug 1277582 might help with the second signature (the HTTP channel one).

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'IPCError-content | (msgtype=0xC2000A,name=???) Route error: message sent to unknown actor ID':
 - nightly (version 51): 18 crashes from 2016-08-01.
 - aurora  (version 50): 915 crashes from 2016-08-01.
 - beta    (version 49): 0 crashes from 2016-08-02.
 - release (version 48): 0 crashes from 2016-07-25.
 - esr     (version 45): 0 crashes from 2016-05-02.

Crash volume on the last weeks (Week N is from 08-22 to 08-28):
            W. N-1  W. N-2  W. N-3
 - nightly       0       4      14
 - aurora      310     225     108
 - beta          0       0       0
 - release       0       0       0
 - esr           0       0       0

Affected platform: Windows

Crash rank on the last 7 days:
             Browser Content     Plugin
 - nightly
 - aurora            #6
 - beta
 - release
 - esr

Comment 5

[[:philipp]]

[Tracking Requested - why for this release]:
this signature is regressing since firefox 50 builds and is currently the #2 crasher in the content process with 10.5% of all crashes in 50.0b1 crash data.

Comment 6

[Benjamin Smedberg]

The other bug 1277582 already landed for 50.0b1 so we're left with a racy-plugin bug here.

PPluginWidget is only for windowed-mode plugins: and because this is a content crash and not a plugin crash, we aren't sure what kind of plugin this is. If this is Flash, this should go away with bug 1305135.

It's also likely possible to do two-phase destruction on this protocol. Let me see if I can do that easily without breaking other stuff.

Comment 7

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Tracking this top crasher as a blocking issue for Fx50.

Comment 8

[Benjamin Smedberg]

Attachment: Bug 1290766 - Return the scroll capture information from the PPluginWidget Create method, instead of using a separate asynchronous method which is delivered later and may race with fast shutdown.

Review commit: https://reviewboard.mozilla.org/r/81500/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/81500/

Comment 9

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Comment on attachment 8795456 [details]
Bug 1290766 - Return the scroll capture information from the PPluginWidget Create method, instead of using a separate asynchronous method which is delivered later and may race with fast shutdown.

https://reviewboard.mozilla.org/r/81500/#review80462

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'IPCError-content | (msgtype=0x8E0017,name=???) Route error: message sent to unknown actor ID':
 - nightly (version 52): 145 crashes from 2016-09-19.
 - aurora  (version 51): 216 crashes from 2016-09-19.
 - beta    (version 50): 583 crashes from 2016-09-20.
 - release (version 49): 0 crashes from 2016-09-05.
 - esr     (version 45): 0 crashes from 2016-06-01.

Crash volume on the last weeks (Week N is from 10-03 to 10-09):
            W. N-1  W. N-2
 - nightly      79      66
 - aurora      184      32
 - beta        447     136
 - release       0       0
 - esr           0       0

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
             Browser Content     Plugin
 - nightly           #22
 - aurora            #10
 - beta              #5
 - release
 - esr

Comment 11

[Benjamin Smedberg]

This patch fails to build on Mac: https://treeherder.mozilla.org/#/jobs?repo=try&revision=4642ac52fdaa

15:36:57     INFO -  In file included from /home/worker/workspace/build/src/widget/PluginWidgetProxy.cpp:8:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/obj-firefox/dist/include/mozilla/plugins/PluginInstanceParent.h:10:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/obj-firefox/ipc/ipdl/_ipdlheaders/mozilla/plugins/PPluginInstanceParent.h:23:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/obj-firefox/dist/include/mozilla/plugins/PluginMessageUtils.h:18:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/obj-firefox/dist/include/npapi.h:37:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/MacOSX10.7.sdk/System/Library/Frameworks/ApplicationServices.framework/Headers/ApplicationServices.h:20:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/MacOSX10.7.sdk/System/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:21:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/MacOSX10.7.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AE.h:20:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/MacOSX10.7.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/CarbonCore.h:20:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/MacOSX10.7.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:38:
15:36:57     INFO -  In file included from /home/worker/workspace/build/src/MacOSX10.7.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFBase.h:108:
15:36:57     INFO -  /home/worker/workspace/build/src/MacOSX10.7.sdk/System/Library/Frameworks/CoreServices.framework/Headers/../Frameworks/CarbonCore.framework/Headers/MacTypes.h:501:16: error: reference to 'Point' is ambiguous
15:36:57     INFO -  typedef struct Point                    Point;
15:36:57     INFO -                 ^
15:36:57     INFO -  /home/worker/workspace/build/src/MacOSX10.7.sdk/System/Library/Frameworks/CoreServices.framework/Headers/../Frameworks/CarbonCore.framework/Headers/MacTypes.h:497:8: note: candidate found by name lookup is 'Point'
15:36:57     INFO -  struct Point {
15:36:57     INFO -         ^
15:36:57     INFO -  /home/worker/workspace/build/src/obj-firefox/dist/include/mozilla/gfx/Point.h:144:34: note: candidate found by name lookup is 'mozilla::gfx::Point'
15:36:57     INFO -  typedef PointTyped<UnknownUnits> Point;

mstange, do you have suggestions for what's going on here or can you maybe pull this and help me fix it? I don't have a mac build environment.

Comment 12

[Markus Stange [:mstange]]

I can take a look in about an hour.

Comment 13

[Markus Stange [:mstange]]

(In reply to Benjamin Smedberg [:bsmedberg] from comment #11)
> mstange, do you have suggestions for what's going on here

PluginWidgetProxy.cpp is being compiled as part of Unified_cpp_widget0.cpp, which includes GfxInfoBase.cpp before it includes PluginWidgetProxy.cpp, and GfxInfoBase.cpp has "using namespace mozilla::gfx;" in it, which makes the type "Point" ambiguous for the rest of Unified_cpp_widget0.cpp.

The easiest way to fix this would probably be to move either PluginWidgetProxy.cpp or GfxInfoBase.cpp from the UNIFIED_SOURCES list in widget/moz.build to the SOURCES list.

Comment 14

[Markus Stange [:mstange]]

Ehsan says it's preferable to move the file that includes the system header to SOURCES, so PluginWidgetProxy.cpp.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/baabb29c04ef
Return the scroll capture information from the PPluginWidget Create method, instead of using a separate asynchronous method which is delivered later and may race with fast shutdown. r=billm

Comment 16

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/baabb29c04ef

Comment 17

[[:philipp]]

should we uplift the patch to aurora/beta? (though the crash volume has gone down after beta 1 and this probably can't be considered a blocking issue anymore)

Comment 18

[Benjamin Smedberg]

Comment on attachment 8795456 [details]
Bug 1290766 - Return the scroll capture information from the PPluginWidget Create method, instead of using a separate asynchronous method which is delivered later and may race with fast shutdown.

Approval Request Comment
[Feature/regressing bug #]: bug 1252877, windowed plugin scrolling improvement
[User impact if declined]: potential crashes when scrolling windowed plugins
[Describe test coverage new/current, TreeHerder]: landed, no regressions found
[Risks and why]: I think this is fairly low risk: it moves a reply to synchronous. *if* async plugin drawing sticks, it's also very low reward since async plugin drawing bypasses this altogether. But right now we're not 100% sure that async plugin drawing will stick.
[String/UUID change made/needed]: none

Comment 19

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8795456 [details]
Bug 1290766 - Return the scroll capture information from the PPluginWidget Create method, instead of using a separate asynchronous method which is delivered later and may race with fast shutdown.

This crash hasn't occurred on Nightly52 since the fix landed so that's a good bit of data, I still see ~100 occurrences on 50.0b5. Let's uplift to Aurora51, Beta50.

Comment 20

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/604ee4191812

Comment 21

[Carsten Book [:Tomcat]]

has problems applying to beta:

grafting 369371:604ee4191812 "Bug 1290766 - Return the scroll capture information from the PPluginWidget Create method, instead of using a separate asynchronous method which is delivered later and may race with fast shutdown. r=billm, a=ritu"
merging dom/plugins/ipc/PluginWidgetChild.cpp
merging dom/plugins/ipc/PluginWidgetParent.cpp
merging widget/PluginWidgetProxy.cpp
merging widget/moz.build
warning: conflicts while merging dom/plugins/ipc/PluginWidgetChild.cpp! (edit, then use 'hg resolve --mark')
abort: unresolved conflicts, can't continue
(use 'hg resolve' and 'hg graft --continue')

Comment 22

[(PTO June 19-July 7) Ryan VanderMeulen]

https://hg.mozilla.org/releases/mozilla-beta/rev/24addbfafd8d

Comment 23

[[:philipp]]

the [@ IPCError-content | (msgtype=0x8E0017,name=???) Route error: message sent to unknown actor ID] signature is still around in 50.0b7 whereas the other variation has ceased. should we file a new report for this or reopen this bug?

Comment 24

[Benjamin Smedberg]

Definitely don't reopen this bug. The patch here fixes PPluginWidget::SetScrollCaptureId and I don't know what the other message is.

Comment 25

[[:philipp]]

ok, i'm removing the signature from this report then.