browser_988072_sidebar_events.js intermittently fails with --e10s --run-by-dir

RESOLVED FIXED in Firefox 37

Status

()

Firefox
Toolbars and Customization
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: jmaher, Assigned: Gijs)

Tracking

(Blocks: 1 bug)

unspecified
Firefox 37
x86_64
Linux
Points:
2
Dependency tree / graph
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(e10s+)

Details

(Reporter)

Description

3 years ago
I am working on adjust --run-by-dir to work for browser-chrome and --e10s, unfortunately a common failure in the bc1 suite is brower_988072_sidebar_events.js:
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=2db7095c9402&searchQuery=e10s-browser

I can easily reproduce this locally on my linux64 desktop:
./mach mochitest-browser --e10s browser/components/customizableui/test

In playing with the manifest file, I can remove any one of these files and the error goes away locally:
[browser_989338_saved_placements_not_resaved.js]
[browser_989751_subviewbutton_class.js]
[browser_987177_xul_wrapper_updating.js]
[browser_987185_syncButton.js]

This is the error I get for the test:
13:13:05 INFO - 539 INFO TEST-START | chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js
13:13:05 INFO - 540 INFO checking window state
13:13:05 INFO - 541 INFO Entering test
13:13:05 INFO - 542 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js | Should have the same number of children - Got 0, expected 2
13:13:05 INFO - Stack trace:
13:13:05 INFO - chrome://mochikit/content/browser-test.js:test_is:834
13:13:05 INFO - chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js:compareList:70
13:13:05 INFO - chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js:null:95
13:13:05 INFO - self-hosted:InterpretGeneratorResume:960
13:13:05 INFO - self-hosted:next:886
13:13:05 INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:651:9
13:13:05 INFO - Tester.prototype.nextTest</<@chrome://mochikit/content/browser-test.js:577:7
13:13:05 INFO - SimpleTest.waitForFocus/maybeRunTests/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:683:49
13:13:05 INFO - *************************
13:13:05 INFO - A coding exception was thrown and uncaught in a Task.
13:13:05 INFO - Full message: TypeError: document.getElementById(...) is null
13:13:05 INFO - Full stack: @chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js:99:3
13:13:05 INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:651:9
13:13:05 INFO - Tester.prototype.nextTest</<@chrome://mochikit/content/browser-test.js:577:7
13:13:05 INFO - SimpleTest.waitForFocus/maybeRunTests/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:683:49
13:13:05 INFO - *************************
13:13:05 INFO - 543 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js | Uncaught exception - at chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js:99 - TypeError: document.getElementById(...) is null
13:13:05 INFO - Stack trace:
13:13:05 INFO - @chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js:99:3
13:13:05 INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:651:9
13:13:05 INFO - Tester.prototype.nextTest</<@chrome://mochikit/content/browser-test.js:577:7
13:13:05 INFO - SimpleTest.waitForFocus/maybeRunTests/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:683:49
13:13:05 INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:651:9
13:13:05 INFO - Tester.prototype.nextTest</<@chrome://mochikit/content/browser-test.js:577:7
13:13:05 INFO - SimpleTest.waitForFocus/maybeRunTests/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:683:49
13:13:09 INFO - 544 INFO TEST-OK | chrome://mochitests/content/browser/browser/components/customizableui/test/browser_988072_sidebar_events.js | took 4064ms 


Is there something I can do to help figure this out?
(Reporter)

Updated

3 years ago
Blocks: 1057512
Does the summary mean that the test fails:

* if you use --e10s and --run-by-dir

or

* if you use *either* --e10s or --run-by-dir ?

I'm confused because this test is already running on tinderbox and passing with e10s, but your commandline doesn't enable run by dir. What do you actually mean?
Flags: needinfo?(jmaher)
(In reply to Joel Maher (:jmaher) from comment #0)
> I am working on adjust --run-by-dir to work for browser-chrome and --e10s,
> unfortunately a common failure in the bc1 suite is
> brower_988072_sidebar_events.js:
> https://treeherder.mozilla.org/ui/#/
> jobs?repo=try&revision=2db7095c9402&searchQuery=e10s-browser
> 
> I can easily reproduce this locally on my linux64 desktop:
> ./mach mochitest-browser --e10s browser/components/customizableui/test
> 
> In playing with the manifest file, I can remove any one of these files and
> the error goes away locally:
> [browser_989338_saved_placements_not_resaved.js]
> [browser_989751_subviewbutton_class.js]

these should be run after this test, so I don't understand how they can influence whether this test fails.
(Reporter)

Comment 3

3 years ago
I had the same confusion on the two tests which run afterwards, but I could reproduce the same error 100% locally, and 3 times in a row with one line removed from the manifest and it would always pass.

To answer the first question, running the command (./mach mochitest-browser --e10s browser/components/customizableui/test) I gave treats it like --run-by-dir (since we are just running a single directory).  Right now we run non e10s browser chrome with --run-by-dir in automation, but e10s is blocked on what appears to be only this.
Flags: needinfo?(jmaher)
(In reply to Joel Maher (:jmaher) from comment #3)
> I had the same confusion on the two tests which run afterwards, but I could
> reproduce the same error 100% locally, and 3 times in a row with one line
> removed from the manifest and it would always pass.

That really doesn't make sense. Did you verify the tests run in order, ie the tests we're confused about are genuinely run after the the test this bug is about?

> To answer the first question, running the command (./mach mochitest-browser
> --e10s browser/components/customizableui/test) I gave treats it like
> --run-by-dir (since we are just running a single directory).  Right now we
> run non e10s browser chrome with --run-by-dir in automation, but e10s is
> blocked on what appears to be only this.

Only this test? rs=me to disable this test itself and land run-by-dir. Just this test shouldn't hold up run-by-dir.
(Reporter)

Comment 5

3 years ago
well, I would rather fix this if we can reasonably.  Let me look at the run order when I am back online in 2-3 hours.  I would like to understand this more before disabling this.

For reference, we turned on run-by-dir (disabling 3 tests) for browser-chrome (non e10s).  As far as I know this is the only remaining test to fix for e10s mode browser-chrome.
(Reporter)

Comment 6

3 years ago
ok, my assertions of which tests were causing the failure were after narrowing down the list to just those tests + sidebar_events; in the big scope of things, I cannot see a fix with any of the above mentioned tests removed from the test list :(
FWIW, I can't reproduce this locally on OS X... can you reproduce locally on any OS? Or is this Linux-only?
Flags: needinfo?(jmaher)
(Reporter)

Comment 8

3 years ago
I only have linux64 available locally and it reproduces easily on my machine.  The predominant errors are on linux, I recall seeing in another push an error on windows.

We disabled this test yesterday to land support for e10s browser-chrome, I would be happy to adjust this down to a linux only disabling!  Likewise try any patches or gather information locally.  I will get my win7 VM up and running again and maybe that could give us additional information.
Flags: needinfo?(jmaher)

Updated

3 years ago
Summary: browser_988072_sidebar_events.js fails in --e10s mode and --run-by-dir → browser_988072_sidebar_events.js fails with --e10s --run-by-dir
Blocks: 984139
tracking-e10s: --- → +
Flags: needinfo?(gijskruitbosch+bugs)
I can't reproduce this on my Linux VM either, which makes this near impossible to debug / figure out in the short term. :-\

If this didn't fail on Windows/Mac, can we enable the test there?

You wrote that you can trivially reproduce on your machine, does it happen if you run the test in isolation, too? If not, any chance I can have a look in Portland? :-)
Flags: needinfo?(gijskruitbosch+bugs) → needinfo?(jmaher)
Summary: browser_988072_sidebar_events.js fails with --e10s --run-by-dir → browser_988072_sidebar_events.js intermittently fails with --e10s --run-by-dir
(Reporter)

Comment 10

3 years ago
I just verified that I could reproduce this on a recent local build on linux64.  I would love to meet up with you in Portland to do this.
Flags: needinfo?(jmaher)
(In reply to :Gijs Kruitbosch from comment #11)
> remote:  
> https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=00f14617f2eb

This looks pretty good considering the retriggers. I'll land it next time I'm near an open tree with a non-empty stomach. :-)
https://hg.mozilla.org/integration/fx-team/rev/c677242e5be8
Assignee: nobody → gijskruitbosch+bugs
Status: NEW → ASSIGNED
Iteration: --- → 37.1
Points: --- → 2
https://hg.mozilla.org/mozilla-central/rev/c677242e5be8
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → Firefox 37
You need to log in before you can comment on or make changes to this bug.