1098386 - talos a11y crashes when run in e10s mode

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

this is one of the few remaining talos tests to get fixed/investigated for e10s mode.  I spent some time looking into this and can easily reproduce the crash on a local (linux64) build of firefox.

This test runs fine while run without e10s, but we seem to crash when we get to the sendAsyncMessage(..) call.  What is odd is that other tests can use this with no problem.

Here is the area in the code that is crashing:
http://hg.mozilla.org/build/talos/file/2bdbc49134c4/talos/page_load_test/a11y/dhtml.html#l42

I am sure there are other things that could be done to narrow this down more, I will count on the expertise of others to help me figure out what else to try.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

actually this line is causing us to fail:
http://hg.mozilla.org/build/talos/file/2bdbc49134c4/talos/page_load_test/a11y/a11y.js#l15
      gAccRetrieval =
      Components.classes["@mozilla.org/accessibleRetrieval;1"]
                .getService(Components.interfaces.nsIAccessibleRetrieval);

I am not familiar with these tests, or how a11y works (or if it should work with e10s), but this is what is getting in the way of us enabling this test for e10s talos.

:davidb, any thoughts on this?

Comment 2

[David Bolter [:davidb] (NeedInfo me for attention)]

Heyo. Can you post a crash stack?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: a11y_e10s_crash.txt

this is the dump from mozcrash, I am using this build (linux64):
https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-linux64/1415912590/firefox-36.0a1.en-US.linux-x86_64.tar.bz2

and this symbols path:
https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-linux64/1415912590/firefox-36.0a1.en-US.linux-x86_64.crashreporter-symbols.zip

here are some instructions for getting talos setup to run locally:
https://wiki.mozilla.org/Buildbot/Talos/Running#Running_locally_-_Source_Code

I would be happy help where ever else I can!

Comment 4

[Trevor Saunders (:tbsaunde)]

(In reply to Joel Maher (:jmaher) from comment #3)
> Created attachment 8522501 [details]
> a11y_e10s_crash.txt
> 
> this is the dump from mozcrash, I am using this build (linux64):
> https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-
> inbound-linux64/1415912590/firefox-36.0a1.en-US.linux-x86_64.tar.bz2

This looks like bug 1097614, but with a better stack and str.

Given the stack I think what's going on is we recieve PDocAccessibleConstructor message while nsAccessibilityService::gAccService is null, but I'm not yet sure how that's possible.

Comment 5

[Trevor Saunders (:tbsaunde)]

Attachment: Allow PDocAccessibleConstructor to be recieved when gAccessibilityService doesn't exist

Talos causes accessibility to be instantiated only in the content
process.  That means PDocAccessibleConstructor messages can be recieved
by the parent when GetAccService() returns null.  The easiest way of
dealing with this is moving DocManager::mRemoteDocuments from a member
of the acc service singleton to its own global.  That means the parent
process can track accessible documents in child processes without
instantiating a11y in the parent process.

Comment 6

[alexander :surkov (:asurkov)]

why to care about managing the remote documents if AccService wasn't instantiated?

Comment 7

[Trevor Saunders (:tbsaunde)]

(In reply to alexander :surkov from comment #6)
> why to care about managing the remote documents if AccService wasn't
> instantiated?

it might be in the future though

Comment 8

[alexander :surkov (:asurkov)]

so you create accessibility related objects in content and/or main processes even if no accessibility running?

Comment 9

[Trevor Saunders (:tbsaunde)]

(In reply to alexander :surkov from comment #8)
> so you create accessibility related objects in content and/or main processes
> even if no accessibility running?

no, if accessibility is running in child but not parent then parent still tracks what documents child has.  How was that not clear from the commit message?

Comment 10

[alexander :surkov (:asurkov)]

how many instances of AccService you have, one per process?

Comment 11

[Trevor Saunders (:tbsaunde)]

(In reply to alexander :surkov from comment #10)
> how many instances of AccService you have, one per process?

yes

Comment 12

[alexander :surkov (:asurkov)]

Comment on attachment 8533451 [details] [diff] [review]
Allow PDocAccessibleConstructor to be recieved when gAccessibilityService doesn't exist

Review of attachment 8533451 [details] [diff] [review]:
-----------------------------------------------------------------

it makes sense, r=me

::: accessible/base/DocManager.h
@@ +80,5 @@
>      MOZ_ASSERT(result, "Why didn't we find the document!");
>    }
>  
>    /*
>     * Notify of a new top level document in a content process.

maybe 'Notify the main process of ...'

@@ +173,5 @@
>      XPCDocumentHashtable;
>    XPCDocumentHashtable mXPCDocumentCache;
>  
>    /*
>     * The list of remote top level documents.

maybe it'd be good to add ', used by (kept in) main process' to make it clear it's not used in content processes.

Comment 13

[Trevor Saunders (:tbsaunde)]

https://bugzilla.mozilla.org/show_bug.cgi?id=1098386

Comment 14

[David Bolter [:davidb] (NeedInfo me for attention)]

ITYM http://hg.mozilla.org/integration/mozilla-inbound/rev/39398fba066e :)

Comment 15

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/97df7d232e7f for leaking in pretty much every test suite:
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=4474474&repo=mozilla-inbound

Comment 16

[Trevor Saunders (:tbsaunde)]

Attachment: Allow PDocAccessibleConstructor to be recieved when gAccessibilityService doesn't exist

Talos causes accessibility to be instantiated only in the content
process.  That means PDocAccessibleConstructor messages can be recieved
by the parent when GetAccService() returns null.  The easiest way of
dealing with this is moving DocManager::mRemoteDocuments from a member
of the acc service singleton to its own global.  That means the parent
process can track accessible documents in child processes without
instantiating a11y in the parent process.

Comment 17

[Trevor Saunders (:tbsaunde)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/0835ea5fd49d

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/0835ea5fd49d