1526914 - Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_execute_sandboxes.py TestExecuteSandboxesChrome.test_execute_new_sandbox | AssertionError: JavascriptException not raised

Status: RESOLVED INCOMPLETE

(Testing :: Marionette Client and Harness, defect, P5)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: hskupin [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=226972037&repo=try

https://queue.taskcluster.net/v1/task/HvN4L3IeRQWqwRq9bkS-TQ/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Here the failure in detail:

[task 2019-02-07T22:26:52.269Z] 22:26:52 WARNING - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_execute_sandboxes.py TestExecuteSandboxesChrome.test_execute_new_sandbox | AssertionError: JavascriptException not raised
[task 2019-02-07T22:26:52.269Z] 22:26:52 INFO - Traceback (most recent call last):
[task 2019-02-07T22:26:52.269Z] 22:26:52 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2019-02-07T22:26:52.269Z] 22:26:52 INFO - testMethod()
[task 2019-02-07T22:26:52.270Z] 22:26:52 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_execute_sandboxes.py", line 52, in test_execute_new_sandbox
[task 2019-02-07T22:26:52.270Z] 22:26:52 INFO - sandbox="1", new_sandbox=True)

Here the source:

https://searchfox.org/mozilla-central/rev/5e3bffe964110b8d1885b4236b8abd5c94d8f609/testing/marionette/harness/marionette_harness/tests/unit/test_execute_sandboxes.py#49-52

I actually don't know why this is failing. It could only mean that there is something different with sandboxes in Fennec.

Geoff, I would be happy to just mark this test as skipped for now so that we can continue moving to the new emulator.

Comment 2

[Geoff Brown [:gbrown]]

I don't have a better idea. Let's skip it.

Comment 3

[Geoff Brown [:gbrown]]

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=f8610a702781ee53c9381c67a945344acf9c0eb1

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This seems to work fine. Lets get the skip patch landed via this bug so we can correctly track the disabled state of the test.

Comment 5

[Geoff Brown [:gbrown]]

Attachment: skip failing test on Android

Comment 6

[Geoff Brown [:gbrown]]

Leave open for follow-up on failing test.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b1e2b327f134
Skip Mn test_execute_new_sandbox on Android; r=me,a=test-only

Comment 8

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/b1e2b327f134

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This is still a problem even with bug 1500509 fixed.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Lets close this intermitten for now given that Fennec tests don't run anymore on central. If it turns out to be still a problem, we could open it again.

Comment 11

[BMO Automation]

Moving bug to Testing::Marionette Client and Harness component per bug 1815831.