1189277 - Crash in IPCError-browser | (msgtype=0x580009,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect, P2)

Description

[Jim Mathies [:jimm]]

This is an outstanding a11y KillHard abort. Looks like the api must be async since the failures happen in during a random mix of unrelated e10s api calls.

Bug 1116884 KillHard child signature breakdown:
----------------------------------------------------
23	51.1%	SendRpcMessage(..)
7	15.6%	mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(..)
3	6.7%	SendSyncMessage(..)
3	6.7%	mozilla::dom::PContentChild::SendGetBlocklistState(..)
2	4.4%	mozilla::net::PCookieServiceChild::SendGetCookieString(..)
2	4.4%	mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,unsigned __int64 const &,mozilla::layers::TargetConfig const &,nsTArray<mozilla::layers::PluginWindowData> const &,bool const &,bool const &,unsigned int const &,bool const &,mozilla::TimeStamp const &,nsTArray<mozilla::layers::EditReply> *)
2	4.4%	mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::IdType<mozilla::dom::TabParent> const &,mozilla::dom::ScreenDetails *,bool *)
1	2.2%	mozilla::dom::PContentChild::SendClipboardHasType(nsTArray<nsCString> const &,int const &,bool *)
1	2.2%	mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const&, unsigned long const&, mozilla::layers::TargetConfig const&, nsTArray<mozilla::layers::PluginWindowData> const&, bool const&, bool const&, unsigned int const&, bool const&, mozilla::TimeStamp const&, nsTArray<mozilla::layers::EditReply>*)
1	2.2%	mozilla::dom::PBrowserChild::SendGetInputContext(..)

IPC error breakdown per KillHard signature:
----------------------------------------------------
SendRpcMessage(..):
  3 - '(msgtype=0x3C0002,name=???) Route error: message sent to unknown actor ID'
  3 - '(msgtype=0x200012,name=???) Message not allowed: cannot be sent/recvd in this state'
  6 - 'HangMonitor'
  8 - '(msgtype=0x200004,name=PBrowser::Msg_PDocAccessibleConstructor) Processing error: message was deserialized, but the handler returned false (indicating failure)'
  4 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::plugins::PPluginWidgetChild::SendGetNativePluginPort(..):
  1 - '(msgtype=0xA20006,name=???) Message not allowed: cannot be sent/recvd in this state'
  7 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

SendSyncMessage(..):
  3 - 'HangMonitor'
  1 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PContentChild::SendGetBlocklistState(..):
  4 - '(msgtype=0x3A004E,name=PContent::Msg_GetBlocklistState) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::net::PCookieServiceChild::SendGetCookieString(..):
  2 - '(msgtype=0x3C0002,name=???) Route error: message sent to unknown actor ID'
  1 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const &,unsigned __int64 const &,mozilla::layers::TargetConfig const &,nsTArray<mozilla::layers::PluginWindowData> const &,bool const &,bool const &,unsigned int const &,bool const &,mozilla::TimeStamp const &,nsTArray<mozilla::layers::EditReply> *):
  3 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PScreenManagerChild::SendScreenForBrowser(mozilla::dom::IdType<mozilla::dom::TabParent> const &,mozilla::dom::ScreenDetails *,bool *):
  2 - 'HangMonitor'
  1 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PContentChild::SendClipboardHasType(nsTArray<nsCString> const &,int const &,bool *):
  2 - '(msgtype=0x3A00A2,name=???) Message not allowed: cannot be sent/recvd in this state'

mozilla::layers::PLayerTransactionChild::SendUpdate(nsTArray<mozilla::layers::Edit> const&, unsigned long const&, mozilla::layers::TargetConfig const&, nsTArray<mozilla::layers::PluginWindowData> const&, bool const&, bool const&, unsigned int const&, bool const&, mozilla::TimeStamp const&, nsTArray<mozilla::layers::EditReply>*):
  2 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

mozilla::dom::PBrowserChild::SendGetInputContext(..):
  2 - '(msgtype=0x4A0008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)'

Comment 1

[Trevor Saunders (:tbsaunde)]

see my last comment in bug 1185726 I guess we should just change it to assert and return true until we find the bug in the child.

Comment 4

[Trevor Saunders (:tbsaunde)]

Attachment: only coalesce reorder events when a previous one for the same target is obsolete

Having one reorder event that superseeds another does not mean that the
dependant mutation events for the first reorder are obsolete.

Comment 5

[alexander :surkov (:asurkov)]

Comment on attachment 8648170 [details] [diff] [review]
only coalesce reorder events when a previous one for the same target is obsolete

Review of attachment 8648170 [details] [diff] [review]:
-----------------------------------------------------------------

r=me with comments addressed

::: accessible/tests/mochitest/treeupdate/test_bug1189277.html
@@ +21,5 @@
> +      this.eventSeq = [];
> +      this.containerNode = getNode("outerDiv");
> +
> +      this.eventSeq.push(new invokerChecker(EVENT_REORDER, this.containerNode));
> +      this.eventSeq.push(new invokerChecker(EVENT_SHOW, "newChildDoc"));

pls add testing for two HIDE events as well

nit, I would prefer styling
this.eventSeq = [
  new invokerChecker(),
  new invokerChecker()
];

@@ +38,5 @@
> +      }
> +
> +      this.finalCheck = function runTest_finalCheck()
> +      {
> +      ok(true, "dummy test");

you don't have to have finalCheck iirc and that fake ok()

Comment 6

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/4d2723bc0bcb

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/4d2723bc0bcb

Comment 8

[Jim Mathies [:jimm]]

Definitely fixed, can we uplift to aurora? 42.0a2 has the same problem.

Comment 9

[Trevor Saunders (:tbsaunde)]

(In reply to Jim Mathies [:jimm] from comment #8)
> Definitely fixed, can we uplift to aurora? 42.0a2 has the same problem.

its not e10s only, and I'm a little concerned about fall out, but I guess its kind of early.  surkov thoughts?

Comment 10

[alexander :surkov (:asurkov)]

(In reply to Trevor Saunders (:tbsaunde) from comment #9)
> (In reply to Jim Mathies [:jimm] from comment #8)
> > Definitely fixed, can we uplift to aurora? 42.0a2 has the same problem.
> 
> its not e10s only, and I'm a little concerned about fall out, but I guess
> its kind of early.  surkov thoughts?

same worries, but taking into account that no known regressions so far and major of testing doesn't happen on nightly then it's probably ok for uplifting.

Comment 11

[Jim Mathies [:jimm]]

Comment on attachment 8648170 [details] [diff] [review]
only coalesce reorder events when a previous one for the same target is obsolete

Approval Request Comment
[Feature/regressing bug #]:
a11y e10s support
[User impact if declined]:
crashes, bug 1116884 incident rates stay elevated
[Describe test coverage new/current, TreeHerder]:
landed on mc on the 19th, no issues this far.
[Risks and why]:
could regress something undetected.
[String/UUID change made/needed]:
none

Comment 12

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8648170 [details] [diff] [review]
only coalesce reorder events when a previous one for the same target is obsolete

We want to fix a11y

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/6209bf0c49e5

Comment 14

[Jim Mathies [:jimm]]

This still shows up on aurora. Opening this back up so we can track on it.

Comment 15

[David Bolter [:davidb] (NeedInfo me for attention)]

Jim is there a place I can look to see if this killhard still happens? (on nightly)

Comment 16

[Jim Mathies [:jimm]]

(In reply to David Bolter [:davidb] from comment #15)
> Jim is there a place I can look to see if this killhard still happens? (on
> nightly)

Currently isn't around - 

http://www.mathies.com/mozilla/client-abort-report-nightly.txt
http://www.mathies.com/mozilla/client-abort-report-aurora.txt
http://www.mathies.com/mozilla/client-abort-report-beta.txt

Comment 17

[Trevor Saunders (:tbsaunde)]

(In reply to Jim Mathies [:jimm] from comment #16)
> (In reply to David Bolter [:davidb] from comment #15)
> > Jim is there a place I can look to see if this killhard still happens? (on
> > nightly)
> 
> Currently isn't around - 

any reason to believe it isn't fixed?

Comment 18

[Guilherme Lima]

e10s is now disabled when acessibility is on, that's probably the reason these crashes are gone.

Before that I got 2 crashes while browsing and opening pages from www.9gag.com, and both have
ipc_channel_error (msgtype=0x560008,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized, but the handler returned false (indicating failure)

Comment 20

[David Bolter [:davidb] (NeedInfo me for attention)]

Still happening in nightly.

Comment 21

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'IPCError-browser | (msgtype=0x560009,name=PDocAccessible::Msg_BindChildDoc) Processing error: message was deserialized,':
 - nightly (version 50): 16 crashes from 2016-06-06.
 - aurora  (version 49): 106 crashes from 2016-06-07.
 - beta    (version 48): 0 crash from 2016-06-06.
 - release (version 47): 0 crash from 2016-05-31.
 - esr     (version 45): 0 crash from 2016-04-07.

Crash volume on the last weeks:
             Week N-1   Week N-2   Week N-3   Week N-4   Week N-5   Week N-6   Week N-7
 - nightly          0          0          0          0          0          3         13
 - aurora          13          9         17         27         10         24          1
 - beta             0          0          0          0          0          0          0
 - release          0          0          0          0          0          0          0
 - esr              0          0          0          0          0          0          0

Affected platforms: Windows, Mac OS X, Linux

Comment 24

[Jim Mathies [:jimm]]

Looks like bug 1270916 may have addressed this.

Comment 25

[Ben Kelly [:bkelly, not reviewing]]

I've been hitting this once a day on FF51 beta.  Jim, any ideas why I would be triggering this on beta?

https://crash-stats.mozilla.com/report/index/bp-2e3d625c-820b-4ccb-b140-18b032161121
https://crash-stats.mozilla.com/report/index/bp-b8ba88cc-9187-46d7-b96f-7e0992161122

Note, I briefly enabled the windows emoji keyboard the other day, but I have it disabled now.  Did that put me into some kind of accessibility mode?  How do I disable that?  (I have e10s forced on.)

Comment 26

[Jim Mathies [:jimm]]

(In reply to Ben Kelly [:bkelly] from comment #25)
> I've been hitting this once a day on FF51 beta.  Jim, any ideas why I would
> be triggering this on beta?
> 
> https://crash-stats.mozilla.com/report/index/bp-2e3d625c-820b-4ccb-b140-
> 18b032161121
> https://crash-stats.mozilla.com/report/index/bp-b8ba88cc-9187-46d7-b96f-
> 7e0992161122
> 
> Note, I briefly enabled the windows emoji keyboard the other day, but I have
> it disabled now.  Did that put me into some kind of accessibility mode?  How
> do I disable that?  (I have e10s forced on.)

Check to see if you have a11y enabled through about:support. You may hit this if you have e10s forced on and you have some sort of touch screen laptop or soft keyboard active.

Comment 27

[Ciprian Georgiu, Desktop QA]

I ran into this issue with latest Aurora 52.0a2 (2016-12-14) while testing touch events in APZ. The crash happened on mashable.com using Dell XPS 12, Windows 10 x64. 
 - https://crash-stats.mozilla.com/report/index/cf796ca5-d569-46e9-8397-bcba02161214

It seems that are still some recent crashes on 52.0a2 from the stats signatures.

Comment 28

[Jacob W. Klein]

Just wanted to chime in to say that I have another "common scenario" that causes this error, I think.
- Using Windows 10 Insider Preview Build 14986
- Using Firefox 50.0.1.0 x86

Sometimes, if I
- Go to https://www.neowin.net
- Hover over the "Facebook Twitter Google+" image in the header
- POOF. "Bad news first: This tab has crashed"

It doesn't happen all the time, and I can't pin down why. I think it has something to do with how many Flash objects I've previously loaded in the session. Not sure.

Regards,
Jacob

Comment 29

[Jacob W. Klein]

Note: I have e10s forced on, with a few extensions installed.

Comment 30

[(No longer employed by Mozilla) Aaron Klotz]

(In reply to Jacob W. Klein from comment #29)
> Note: I have e10s forced on, with a few extensions installed.

That's the price of force-enabling e10s, I'm afraid. We are currently focusing on 52-53 for stabilizing touch and a11y on e10s.