1162008 - dom/media/webspeech/recognition/test/ crashes when run by directory

Status: RESOLVED WORKSFORME

(Core :: Audio/Video, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

with the exception of windows, test_timeout.html fails pretty consistently when it comes to running a single directory.  On windows test_success_without_recognition_service.html fails.  The pattern, these are the last test in the directory to run.

Here is a view on treeherder from a try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=72b7fdf2885a&filter-searchStr=mochitest-3

and I was able to reproduce this locally on a linux64 environment with a local build of inbound by running:
 ./mach mochitest-plain dom/media/webspeech/recognition/

^ note: this reproduced about 50% of the time locally.

Typically when we are running tests in a runbydir fashion there are 3 reasons for failure:
1) the test depends on a startup state which hasn't happened yet because we are running it much closer to the startup of the browser
2) the test depends on features from other tests that used to run before it
3) the test doesn't clean up and depends on the browser running for a longer duration to do the cleanup

in this case, I am calling out #3 as the culprit.  Now to find a developer that knows anything about this.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

this is the commit history for test_timeout.html:
https://hg.mozilla.org/mozilla-central/filelog/ba44099cbd07/dom/media/webspeech/recognition/test/test_timeout.html

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

what is odd is that windows xp debug doesn't crash at all- maybe that is the best OS ever?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

:smaug, can you take a look at this?  I would prefer not to disable the entire dom/media/webspeech set of tests to solve this problem.  Quite possibly you know who could be a better point of contact.

Comment 4

[Olli Pettay [:smaug][[e-mail]]]

./mach mochitest-plain dom/media/webspeech/recognition/ works fine here, 64bit Fedora 20.
Run that several times.

which crash are you talking about? I see some shutdown issue, coming from webspeech/synth
(how from that directory?), which indeed could hint about 3).


Andre, could you test this locally?

Comment 5

[Olli Pettay [:smaug][[e-mail]]]

(In reply to Olli Pettay [:smaug] from comment #4)
> which crash are you talking about? I see some shutdown issue, coming from
> webspeech/synth
I mean in your tryserver logs.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

there are a few errors in the logs, mostly all the mochitest-3 failures are in recognition or synth (in the cases where the recognition stuff might not run all the tests due to platform restrictions in the manifest)

I really don't understand all the errors, but I would be happy to help where possible.  The crashes appear to happen when we are shutting down, what I saw locally is that we hung there and then the automation would kill it causing a crash.

Comment 7

[André Natal]

(In reply to Olli Pettay [:smaug] from comment #4)
> ./mach mochitest-plain dom/media/webspeech/recognition/ works fine here,
> 64bit Fedora 20.
> Run that several times.
> 
> which crash are you talking about? I see some shutdown issue, coming from
> webspeech/synth
> (how from that directory?), which indeed could hint about 3).
> 
> 
> Andre, could you test this locally?

I had no problem in Ubuntu 14.04 64 bits. I also ran it several times and never had any issue.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

I have been trying to find which test are problematic, for linux, it is [test_timeout.html].

here is a try server showing the run with a linux mochitest-3 crash:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=52e268ea47ae&filter-searchStr=linux

windows is a different beast.  In opt mode we get crashes.  It appears to be the last test in the list:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0da7845ceb4a&filter-searchStr=win
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d5ffffa8ae90&filter-searchStr=win
https://treeherder.mozilla.org/#/jobs?repo=try&revision=27163f70b0d0&filter-searchStr=win

you can see in each of these runs I keep getting a different crash- the thing in common is the crashing test is always the last test ran!

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

thanks for giving this a try locally.

it seems there are 2 issues here (at least identified so far):
* linux (at least 32 bit) crashes on test_timeout.html
* windows* opt crashes on the last test to run.

hmm, speaking of crashes- these might not be crashes of the browser, maybe forced crashes by the test harness due to failure to terminate the browser after completing the test.

That would make a lot more sense.  Is there something in the webspeech/recognition code that might take a while to unregister or cleanup?  we could be locked on it when trying to shutdown right after using it.

Comment 10

[Olli Pettay [:smaug][[e-mail]]]

kdavis, could you take a look at this?

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

:kaustabh, is this still a problem?

Comment 12

[Kaustabh Datta Choudhury]

A push was made to try with these tests enabled, and we got fairly green results : https://treeherder.mozilla.org/#/jobs?repo=try&revision=351ba2066462