Closed Bug 1519552 Opened 6 years ago Closed 5 years ago

Perma Tier2 testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic | AssertionError: {u'value': {u'status': u'PASS', u'message': u'Testing about:blank == about:blank [truncated]... != {u'value':...

Categories

(Testing :: Marionette Client and Harness, defect, P5)

Product:
Testing
Component:
Marionette Client and Harness
Version:
Version 3
Platform:
Unspecified
Android
Type:
defect

Tracking

(firefox65 unaffected, firefox66 disabled)

Status:
RESOLVED WONTFIX
Tracking Flags:
Tracking Status
firefox65 --- unaffected
firefox66 --- disabled

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure, test-disabled, Whiteboard: [stockwell needswork])

Attachments

(1 file)

Bug 1519552 - Disable test_reftest.py on Android
47 bytes, text/x-phabricator-request
Details | Review

Filed by: aiakab [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=221381695&repo=autoland

https://queue.taskcluster.net/v1/task/fNGpDEm7R4Ol8lGmh5qzRA/runs/0/artifacts/public/logs/live_backing.log

[task 2019-01-11T21:28:34.308Z] 21:28:34 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic
[task 2019-01-11T21:28:53.451Z] 21:28:53 WARNING - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic | AssertionError: {u'value': {u'status': u'PASS', u'message': u'Testing about:blank == about:blank [truncated]... != {u'value': {u'status': u'TIMEOUT', u'message': None, u'extra': {}}}
[task 2019-01-11T21:28:53.451Z] 21:28:53 INFO - + {u'value': {u'extra': {}, u'message': None, u'status': u'TIMEOUT'}}
[task 2019-01-11T21:28:53.452Z] 21:28:53 INFO - - {u'value': {u'extra': {},
[task 2019-01-11T21:28:53.452Z] 21:28:53 INFO - - u'message': u'Testing about:blank == about:blank\n',
[task 2019-01-11T21:28:53.452Z] 21:28:53 INFO - - u'stack': None,
[task 2019-01-11T21:28:53.453Z] 21:28:53 INFO - - u'status': u'PASS'}}
[task 2019-01-11T21:28:53.453Z] 21:28:53 INFO - Traceback (most recent call last):
[task 2019-01-11T21:28:53.455Z] 21:28:53 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2019-01-11T21:28:53.455Z] 21:28:53 INFO - testMethod()
[task 2019-01-11T21:28:53.455Z] 21:28:53 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py", line 47, in test_basic
[task 2019-01-11T21:28:53.455Z] 21:28:53 INFO - self.assertEqual(expected, rv)
[task 2019-01-11T21:28:53.455Z] 21:28:53 INFO - TEST-INFO took 19143ms
[task 2019-01-11T21:28:58.419Z] 21:28:58 INFO - test_end for testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic logged while not in progress. Logged with data: {"status": "ERROR", "extra": {"class_name": "test_reftest.TestReftest", "source": "<html><head></head><body></body></html>", "method_name": "test_basic", "screenshot": "iVBORw0KGgoAAAANSUhEUgAABLAAAASwCAYAAADrIbPPAAAgAElEQVR4nOzYMQEAMAyAsPo33TrYO45EATezAAAAABA2vwMAAAAA4MXAAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcC6duxYAAAAAGCQv/U0dhRGAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAXrcgAAAADQSURBVAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWAogrPYAICTjAAAAAAElFTkSuQmCC"}, "expected": "PASS", "test": "testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic", "message": "NoSuchWindowException: Unable to locate window: 16\nstacktrace:\n\tWebDriverError@chrome://marionette/content/error.js:179:5\n\tNoSuchWindowError@chrome://marionette/content/error.js:411:5\n\tGeckoDriver.prototype.switchToWindow@chrome://marionette/content/driver.js:1527:11\n\tdespatch@chrome://marionette/content/server.js:290:20\n\texecute@chrome://marionette/content/server.js:263:11\n\tonPacket/<@chrome://marionette/content/server.js:236:15\n\tonPacket@chrome://marionette/content/server.js:235:8\n\t_onJSONObjectReady/<@chrome://marionette/content/transport.js:493:9", "stack": "Traceback (most recent call last):\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 190, in run\n self.tearDown()\n File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py", line 28, in tearDown\n self.marionette.switch_to_window(self.original_window)\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1522, in switch_to_window\n {"focus": focus, "name": window_id})\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 26, in _\n return func(*args, **kwargs)\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 764, in _send_message\n self._handle_error(err)\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 784, in _handle_error\n raise errors.lookup(error)(message, stacktrace=stacktrace)\n"}
[task 2019-01-11T21:28:58.844Z] 21:28:58 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_cache_multiple_sizes
[task 2019-01-11T21:36:13.352Z] 21:36:13 WARNING - TEST-UNEXPECTED-ERROR | testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_cache_multiple_sizes | IOError: Process killed because the connection to Marionette server is lost. Check gecko.log for errors (Reason: Connection timed out after 360.0s)
[task 2019-01-11T21:36:13.352Z] 21:36:13 INFO - Traceback (most recent call last):
[task 2019-01-11T21:36:13.353Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 140, in run
[task 2019-01-11T21:36:13.354Z] 21:36:13 INFO - self.setUp()
[task 2019-01-11T21:36:13.354Z] 21:36:13 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py", line 13, in setUp
[task 2019-01-11T21:36:13.354Z] 21:36:13 INFO - super(TestReftest, self).setUp()
[task 2019-01-11T21:36:13.354Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 331, in setUp
[task 2019-01-11T21:36:13.355Z] 21:36:13 INFO - super(MarionetteTestCase, self).setUp()
[task 2019-01-11T21:36:13.355Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 254, in setUp
[task 2019-01-11T21:36:13.355Z] 21:36:13 INFO - self.marionette.start_session()
[task 2019-01-11T21:36:13.356Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 26, in _
[task 2019-01-11T21:36:13.356Z] 21:36:13 INFO - return func(*args, **kwargs)
[task 2019-01-11T21:36:13.356Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1282, in start_session
[task 2019-01-11T21:36:13.357Z] 21:36:13 INFO - resp = self._send_message("WebDriver:NewSession", capabilities)
[task 2019-01-11T21:36:13.357Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 36, in _
[task 2019-01-11T21:36:13.357Z] 21:36:13 INFO - m._handle_socket_failure()
[task 2019-01-11T21:36:13.357Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 844, in _handle_socket_failure
[task 2019-01-11T21:36:13.358Z] 21:36:13 INFO - reraise(IOError, message.format(returncode=returncode, reason=val), tb)
[task 2019-01-11T21:36:13.358Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 26, in _
[task 2019-01-11T21:36:13.359Z] 21:36:13 INFO - return func(*args, **kwargs)
[task 2019-01-11T21:36:13.359Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 756, in _send_message
[task 2019-01-11T21:36:13.359Z] 21:36:13 INFO - msg = self.client.request(name, params)
[task 2019-01-11T21:36:13.360Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/transport.py", line 273, in request
[task 2019-01-11T21:36:13.360Z] 21:36:13 INFO - return self.receive()
[task 2019-01-11T21:36:13.360Z] 21:36:13 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/transport.py", line 183, in receive
[task 2019-01-11T21:36:13.361Z] 21:36:13 INFO - raise socket.timeout("Connection timed out after {}s".format(self.socket_timeout))
[task 2019-01-11T21:36:13.361Z] 21:36:13 INFO - TEST-INFO took 434508ms

Summary: Tier2 Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic | AssertionError: {u'value': {u'status': u'PASS', u'message': u'Testing about:blank == about:blank [truncated]... != {u'value':... → Perma Tier2 Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic | AssertionError: {u'value': {u'status': u'PASS', u'message': u'Testing about:blank == about:blank [truncated]... != {u'value':...
Flags: needinfo?(geoffers+mozilla)

There are 78 total failures in the last 7 days all on android-em-4-3-armv7-api16.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=221613048&repo=autoland&lineNumber=1778

[task 2019-01-13T20:32:44.414Z] 20:32:44 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic
[task 2019-01-13T20:33:03.576Z] 20:33:03 WARNING - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic | AssertionError: {u'value': {u'status': u'PASS', u'message': u'Testing about:blank == about:blank [truncated]... != {u'value': {u'status': u'TIMEOUT', u'message': None, u'extra': {}}}
[task 2019-01-13T20:33:03.577Z] 20:33:03 INFO - + {u'value': {u'extra': {}, u'message': None, u'status': u'TIMEOUT'}}
[task 2019-01-13T20:33:03.577Z] 20:33:03 INFO - - {u'value': {u'extra': {},
[task 2019-01-13T20:33:03.577Z] 20:33:03 INFO - - u'message': u'Testing about:blank == about:blank\n',
[task 2019-01-13T20:33:03.578Z] 20:33:03 INFO - - u'stack': None,
[task 2019-01-13T20:33:03.578Z] 20:33:03 INFO - - u'status': u'PASS'}}
[task 2019-01-13T20:33:03.578Z] 20:33:03 INFO - Traceback (most recent call last):
[task 2019-01-13T20:33:03.580Z] 20:33:03 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2019-01-13T20:33:03.580Z] 20:33:03 INFO - testMethod()
[task 2019-01-13T20:33:03.580Z] 20:33:03 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py", line 47, in test_basic
[task 2019-01-13T20:33:03.580Z] 20:33:03 INFO - self.assertEqual(expected, rv)
[task 2019-01-13T20:33:03.580Z] 20:33:03 INFO - TEST-INFO took 19162ms
[task 2019-01-13T20:33:09.838Z] 20:33:09 INFO - test_end for testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic logged while not in progress. Logged with data: {"status": "ERROR", "extra": {"class_name": "test_reftest.TestReftest", "source": "<html><head></head><body></body></html>", "method_name": "test_basic", "screenshot": "iVBORw0KGgoAAAANSUhEUgAABLAAAASwCAYAAADrIbPPAAAgAElEQVR4nOzYMQEAMAyAsPo33TrYO45EATezAAAAABA2vwMAAAAA4MXAAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcACAAAAIM3AAgAAACDNwAIAAAAgzcC6duxYAAAAAGCQv/U0dhRGAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAXrcgAAAADQSURBVAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWBBYAAAAAawILAAAAgDWBBQAAAMCawAIAAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYE1gAAAAArAksAAAAANYEFgAAAABrAgsAAACANYEFAAAAwJrAAgAAAGBNYAEAAACwJrAAAAAAWBNYAAAAAKwJLAAAAADWAogrPYAICTjAAAAAAElFTkSuQmCC"}, "expected": "PASS", "test": "testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic", "message": "NoSuchWindowException: Unable to locate window: 16\nstacktrace:\n\tWebDriverError@chrome://marionette/content/error.js:179:5\n\tNoSuchWindowError@chrome://marionette/content/error.js:411:5\n\tGeckoDriver.prototype.switchToWindow@chrome://marionette/content/driver.js:1527:11\n\tdespatch@chrome://marionette/content/server.js:290:20\n\texecute@chrome://marionette/content/server.js:263:11\n\tonPacket/<@chrome://marionette/content/server.js:236:15\n\tonPacket@chrome://marionette/content/server.js:235:8\n\t_onJSONObjectReady/<@chrome://marionette/content/transport.js:493:9", "stack": "Traceback (most recent call last):\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 190, in run\n self.tearDown()\n File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py", line 28, in tearDown\n self.marionette.switch_to_window(self.original_window)\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1522, in switch_to_window\n {"focus": focus, "name": window_id})\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 26, in _\n return func(*args, **kwargs)\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 764, in _send_message\n self._handle_error(err)\n File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 784, in _handle_error\n raise errors.lookup(error)(message, stacktrace=stacktrace)\n"}
[task 2019-01-13T20:33:10.235Z] 20:33:10 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_cache_multiple_sizes
[task 2019-01-13T20:40:25.327Z] 20:40:25 WARNING - TEST-UNEXPECTED-ERROR | testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_cache_multiple_sizes | IOError: Process killed because the connection to Marionette server is lost. Check gecko.log for errors (Reason: Connection timed out after 360.0s)
[task 2019-01-13T20:40:25.328Z] 20:40:25 INFO - Traceback (most recent call last):
[task 2019-01-13T20:40:25.328Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 140, in run
[task 2019-01-13T20:40:25.329Z] 20:40:25 INFO - self.setUp()
[task 2019-01-13T20:40:25.329Z] 20:40:25 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py", line 13, in setUp
[task 2019-01-13T20:40:25.329Z] 20:40:25 INFO - super(TestReftest, self).setUp()
[task 2019-01-13T20:40:25.330Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 331, in setUp
[task 2019-01-13T20:40:25.330Z] 20:40:25 INFO - super(MarionetteTestCase, self).setUp()
[task 2019-01-13T20:40:25.331Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 254, in setUp
[task 2019-01-13T20:40:25.331Z] 20:40:25 INFO - self.marionette.start_session()
[task 2019-01-13T20:40:25.331Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 26, in _
[task 2019-01-13T20:40:25.332Z] 20:40:25 INFO - return func(*args, **kwargs)
[task 2019-01-13T20:40:25.332Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1282, in start_session
[task 2019-01-13T20:40:25.332Z] 20:40:25 INFO - resp = self._send_message("WebDriver:NewSession", capabilities)
[task 2019-01-13T20:40:25.333Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 36, in _
[task 2019-01-13T20:40:25.333Z] 20:40:25 INFO - m._handle_socket_failure()
[task 2019-01-13T20:40:25.334Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 844, in _handle_socket_failure
[task 2019-01-13T20:40:25.334Z] 20:40:25 INFO - reraise(IOError, message.format(returncode=returncode, reason=val), tb)
[task 2019-01-13T20:40:25.335Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 26, in _
[task 2019-01-13T20:40:25.335Z] 20:40:25 INFO - return func(*args, **kwargs)
[task 2019-01-13T20:40:25.335Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 756, in _send_message
[task 2019-01-13T20:40:25.336Z] 20:40:25 INFO - msg = self.client.request(name, params)
[task 2019-01-13T20:40:25.336Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/transport.py", line 273, in request
[task 2019-01-13T20:40:25.337Z] 20:40:25 INFO - return self.receive()
[task 2019-01-13T20:40:25.337Z] 20:40:25 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/transport.py", line 183, in receive
[task 2019-01-13T20:40:25.338Z] 20:40:25 INFO - raise socket.timeout("Connection timed out after {}s".format(self.socket_timeout))
[task 2019-01-13T20:40:25.338Z] 20:40:25 INFO - TEST-INFO took 435093ms

Whiteboard: [stockwell needswork]
OS: Unspecified → Android
See Also: → 1519614
Summary: Perma Tier2 Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic | AssertionError: {u'value': {u'status': u'PASS', u'message': u'Testing about:blank == about:blank [truncated]... != {u'value':... → Perma Tier2 testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py TestReftest.test_basic | AssertionError: {u'value': {u'status': u'PASS', u'message': u'Testing about:blank == about:blank [truncated]... != {u'value':...

Patch up to just address this by disabling this tests on Android for now. The reftests seem to run without problem on Android, which suggest something is up with how we're running these tests on Android rather than the code under test in these unit tests.

Assignee: nobody → geoffers+mozilla
Flags: needinfo?(james)
Flags: needinfo?(geoffers+mozilla)
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/701665c38279
Disable test_reftest.py on Android r=jgraham,whimboo

https://hg.mozilla.org/mozilla-central/rev/701665c38279

Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox66: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla66
Status: RESOLVED → REOPENED
status-firefox66: fixed → disabled
Keywords: leave-open, test-disabled
Resolution: FIXED → ---
Target Milestone: mozilla66 → ---
Assignee: geoffers+mozilla → nobody
Blocks: 1578783

Geoff, we won't run this test for any GeckoView application. So when you remove the skip it should be all fine.

James, I'm not sure how the future of wptrunner looks like, but we cannot run this test with Marionette for GeckoView. If it's not worth running the test on that platform, we should just close this bug.

https://searchfox.org/mozilla-central/source/testing/marionette/harness/marionette_harness/tests/unit/test_reftest.py

Flags: needinfo?(james)

Can you explain what the problem is with running it on Android? I expect it doesn't matter much since it's generally fairly obvious when reftests break, but some reasoning would be helpful. Is marionette harness just not working on geckoview at all?

Flags: needinfo?(james)

(In reply to James Graham [:jgraham] from comment #15)

Can you explain what the problem is with running it on Android? I expect it doesn't matter much since it's generally fairly obvious when reftests break, but some reasoning would be helpful. Is marionette harness just not working on geckoview at all?

That's correct. The harness doesn't support geckoview, and we are not going to get this added. As such all Marionette unit tests are no longer running on central for Android. Ultimately geckodriver will be used for wdspec, and remaining Marionette unit tests will be ported over. Not sure if that would work for those reftests.

If that's fine with you we can close this bug as wontfix.

Flags: needinfo?(james)

Yes, that's fine. These endpoints aren't exposed through geckodriver afaik, but we obviously have wpt infrastructure tests to ensure reftests keep working on android in general.

Flags: needinfo?(james)

Perfect. So the patch on bug 1578783 removed the skip flag for Fennec.

Status: REOPENED → RESOLVED
Closed: 6 years ago → 5 years ago
Keywords: leave-open
Resolution: --- → WONTFIX
Product: Testing → Remote Protocol
Moving bug to Testing::Marionette Client and Harness component per bug 1815831.
Component: Marionette → Marionette Client and Harness
Product: Remote Protocol → Testing
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: