1095723 - browser_addons.js fails quite frequently in e10s and --run-by-dir

Status: RESOLVED FIXED

(Firefox Graveyard :: SocialAPI, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

in bug 1057512 we are looking to run tests per directory instead of a large chunk.  This causes some tests to fail that depend on services being setup by an earlier test (or time). here is a try server push:
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=24074e9f7d38&searchQuery=e10s-browser-chrome-1

10:43:51 INFO - 427 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/base/content/test/social/browser_addons.js | Test timed out - expected PASS
10:43:51 INFO - 428 INFO TEST-OK | chrome://mochitests/content/browser/browser/base/content/test/social/browser_addons.js | took 90064ms
10:43:51 INFO - 429 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/base/content/test/social/browser_addons.js | Found a tab after previous test timed out: https://test1.example.com/browser/browser/base/content/test/social/social_activate.html - expected PASS

Comment 1

[Shane Caraveo (:mixedpuppy)]

Socialapi tests are all independent.  This is probably caused by bug 1095457.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

I pushed to try with the patch from bug 1095457 and it didn't help.  Maybe this is related to time the browser has been running?  I know a lot of services are doing delayed startup, browser_addons.js is the first test to run, so it could be that we need to do something else until all services are properly intialized.

Comment 3

[Shane Caraveo (:mixedpuppy)]

(In reply to Joel Maher (:jmaher) from comment #2)
> I pushed to try with the patch from bug 1095457 and it didn't help.  Maybe
> this is related to time the browser has been running?  I know a lot of
> services are doing delayed startup, browser_addons.js is the first test to
> run, so it could be that we need to do something else until all services are
> properly intialized.

The init of socialapi does start in delayedStartup and after session restore. I'm not sure how/when tests are started, but if they start before session restore that could be an issue (in more than just socialapi). Looking at the code in browser.js, I'm also uncertain whether the browser-delayed-startup-finished notification would be late enough.

This error at the start of the tests in the social directory feels suspect (not saying it's the cause, but a concern?)

10:04:41     INFO -  testpath: browser/base/content/test/social
10:04:42     INFO -  380 ERROR Got second suite_start message before suite_end. Logged with data ...


I'll try this on osx today and see if I can repro, otherwise will try it on linux tomorrow.

Comment 4

[Shane Caraveo (:mixedpuppy)]

well, osx was fine.  when I run on linux, I get an ssl cert problem and I'm not sure if that is some config issue on my vm, or a general problem, but it happens in the same place you're timing out:


./mach mochitest-browser --e10s --run-by-dir browser/base/content/test/social/

...

30 INFO sub-test testForeignInstall starting
31 INFO Waiting for install dialog
32 INFO Console message: [JavaScript Error: "test1.example.com:443 uses an invalid security certificate.

The certificate will not be valid until 10/01/2014 02:21 PM. The current time is 08/22/2014 07:11 PM.

(Error code: sec_error_expired_certificate)
"]

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

it looks like the time on the linux vm is august, ssl will fail then :)

Comment 6

[Shane Caraveo (:mixedpuppy)]

/me facepalm  

ok, this passes locally on linux, but in order to do so I have to click on the firefox window to bring it to front before that test occurs.  That in particular is a recurring issue I have running mochi tests on linux, the test results window ends up on top, and any ui interactions fail.

Comment 7

[Shane Caraveo (:mixedpuppy)]

(In reply to Shane Caraveo (:mixedpuppy) from comment #6)
> /me facepalm  
> 
> ok, this passes locally on linux, but in order to do so I have to click on
> the firefox window to bring it to front before that test occurs.  That in
> particular is a recurring issue I have running mochi tests on linux, the
> test results window ends up on top, and any ui interactions fail.

and... http://mozilla-releng-blobs.s3.amazonaws.com/blobs/try/sha512/9d8511a6772dbc45bf473336692b79b9ca760fd2b763031730610b9faebce64f5cc2f8c09b82b52917144122b88cf1fa979d4564767d2eaf5b0f730cf021b51f

Looks to me like the test server has the same problem.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

https://tbpl.mozilla.org/?tree=Try&rev=d7185987036b

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: set focus to test window for social tests (1.0)

try server seems to think this is safe for regular mode:
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=d7185987036b

testing locally seems to think it works in run-by-dir with e10s!

Comment 10

[Shane Caraveo (:mixedpuppy)]

Comment on attachment 8520973 [details] [diff] [review]
set focus to test window for social tests (1.0)

cool, I had tried focusing the window in testing/mochitest/browser-test.js but it didn't work from there.

Comment 11

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/integration/fx-team/rev/0ccf36457e3b

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/0ccf36457e3b