Closed Bug 1411348 Opened 5 years ago Closed 5 years ago

Permafail TestCrash.test_crash_chrome_process | AssertionError: None unexpectedly found in (None, 0)


(Testing :: Marionette, defect)

Version 3
Not set


(firefox58 fixed)

Tracking Status
firefox58 --- fixed


(Reporter: aryx, Assigned: gbrown)



(Keywords: intermittent-failure, Whiteboard: [stockwell disabled])


(1 file)

This started to permafail on Android 4.3 API 16+ debug when the patch for bug 1410355 landed:

Failure log:
[task 2017-10-23T17:34:46.691Z] 17:34:46     INFO -  TEST-START | TestCrash.test_crash_chrome_process
[task 2017-10-23T17:34:52.582Z] 17:34:52  WARNING -  TEST-UNEXPECTED-FAIL | TestCrash.test_crash_chrome_process | AssertionError: None unexpectedly found in (None, 0)
[task 2017-10-23T17:34:52.582Z] 17:34:52     INFO -  Traceback (most recent call last):
[task 2017-10-23T17:34:52.583Z] 17:34:52     INFO -    File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/", line 156, in run
[task 2017-10-23T17:34:52.583Z] 17:34:52     INFO -      testMethod()
[task 2017-10-23T17:34:52.585Z] 17:34:52     INFO -    File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/", line 123, in test_crash_chrome_process
[task 2017-10-23T17:34:52.585Z] 17:34:52     INFO -      self.assertNotIn(self.marionette.instance.runner.returncode, (None, 0))
[task 2017-10-23T17:34:52.585Z] 17:34:52     INFO -  TEST-INFO took 5891ms
[task 2017-10-23T17:34:53.377Z] 17:34:53     INFO -  mozdevice Non-zero return code (1) from ['/builds/worker/workspace/build/android-sdk-linux/platform-tools/adb', '-s', 'emulator-5554', 'forward', '--remove', 'tcp:2828']
[task 2017-10-23T17:34:53.378Z] 17:34:53     INFO -  mozdevice Output: ["error: listener 'tcp:2828' not found"]
[task 2017-10-23T17:34:53.380Z] 17:34:53 CRITICAL -  test_end for TestCrash.test_crash_chrome_process logged while not in progress. Logged with data: {"status": "ERROR", "extra": {"class_name": "test_crash.TestCrash", "method_name": "test_crash_chrome_process"}, "expected": "PASS", "test": " TestCrash.test_crash_chrome_process", "message": "DMError: Failed to remove connection forwarding.", "stack": "Traceback (most recent call last):\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/\", line 187, in run\n    self.tearDown()\n  File \"/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/\", line 87, in tearDown\n    super(BaseCrashTestCase, self).tearDown()\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/\", line 322, in tearDown\n    self.marionette.start_session()\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/\", line 28, in _\n    m._handle_socket_failure()\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/\", line 818, in _handle_socket_failure\n    self.cleanup()\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/\", line 655, in cleanup\n    self.instance.close(clean=True)\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/\", line 407, in close\n    \"tcp:{}\".format(self.marionette_port))\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/mozdevice/\", line 211, in remove_forward\n    raise DMError(\"Failed to remove connection forwarding.\")\n"}
This test should not be run on Android. Maybe the skip logic is wrong. I will have a look at it tomorrow.
Flags: needinfo?(hskupin)
I think the test for fennec is okay, but there's just a single '|' earlier in the annotation.
Attachment #8922366 - Flags: review?(hskupin)
Comment on attachment 8922366 [details] [diff] [review]
fix test annotation

Review of attachment 8922366 [details] [diff] [review]:

Ups! Sorry for that mistake. Thanks for fixing it.
Attachment #8922366 - Flags: review?(hskupin) → review+
Pushed by
Fix manifest annotation for; r=whimboo
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla58
Whiteboard: [stockwell disabled]
Joel, this test is not disabled per-se. But only for those platforms it cannot be run on. As such I remove your whiteboard entry again.
Flags: needinfo?(hskupin)
Whiteboard: [stockwell disabled]
:whimboo, how would you categorize the fix?  If this doesn't run on Android, we disabled the test from running there to fix the problem, correct?  I do not see a fix to the product or a fix to the test in order to see this resolved.
Flags: needinfo?(hskupin)
This test cannot be run on Android because the application is not controlled by Marionette. The fix here was only a regression from by patch on bug 1410355, and didn't change anything in regards of where the tests are getting run.
Flags: needinfo?(hskupin)
given this the problem of a perma failing test was resolved by disabling it on the platform where it was perma failing.  We have marked many tests as disabled for turning off jobs where they are not wanted anymore, etc.
Whiteboard: [stockwell disabled]
Assignee: nobody → gbrown
You need to log in before you can comment on or make changes to this bug.