Closed Bug 1411348 Opened 7 years ago Closed 7 years ago

Permafail test_crash.py TestCrash.test_crash_chrome_process | AssertionError: None unexpectedly found in (None, 0)

Categories

(Testing :: Marionette Client and Harness, defect)

Product:
Testing
Component:
Marionette Client and Harness
Version:
Version 3
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox58 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla58
Tracking Flags:
Tracking Status
firefox58 --- fixed

People

(Reporter: aryx, Assigned: gbrown)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell disabled])

Attachments

(1 file)

fix test annotation
755 bytes, patch
whimboo
: review+
Details | Diff | Splinter Review 
This started to permafail on Android 4.3 API 16+ debug when the patch for bug 1410355 landed: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=363bf0516c8938219c67553522f9e6ac69b98fad&selectedJob=138945618

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=138945618&repo=autoland
[task 2017-10-23T17:34:46.691Z] 17:34:46     INFO -  TEST-START | test_crash.py TestCrash.test_crash_chrome_process
[task 2017-10-23T17:34:52.582Z] 17:34:52  WARNING -  TEST-UNEXPECTED-FAIL | test_crash.py TestCrash.test_crash_chrome_process | AssertionError: None unexpectedly found in (None, 0)
[task 2017-10-23T17:34:52.582Z] 17:34:52     INFO -  Traceback (most recent call last):
[task 2017-10-23T17:34:52.583Z] 17:34:52     INFO -    File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 156, in run
[task 2017-10-23T17:34:52.583Z] 17:34:52     INFO -      testMethod()
[task 2017-10-23T17:34:52.585Z] 17:34:52     INFO -    File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_crash.py", line 123, in test_crash_chrome_process
[task 2017-10-23T17:34:52.585Z] 17:34:52     INFO -      self.assertNotIn(self.marionette.instance.runner.returncode, (None, 0))
[task 2017-10-23T17:34:52.585Z] 17:34:52     INFO -  TEST-INFO took 5891ms
[task 2017-10-23T17:34:53.377Z] 17:34:53     INFO -  mozdevice Non-zero return code (1) from ['/builds/worker/workspace/build/android-sdk-linux/platform-tools/adb', '-s', 'emulator-5554', 'forward', '--remove', 'tcp:2828']
[task 2017-10-23T17:34:53.378Z] 17:34:53     INFO -  mozdevice Output: ["error: listener 'tcp:2828' not found"]
[task 2017-10-23T17:34:53.380Z] 17:34:53 CRITICAL -  test_end for test_crash.py TestCrash.test_crash_chrome_process logged while not in progress. Logged with data: {"status": "ERROR", "extra": {"class_name": "test_crash.TestCrash", "method_name": "test_crash_chrome_process"}, "expected": "PASS", "test": "test_crash.py TestCrash.test_crash_chrome_process", "message": "DMError: Failed to remove connection forwarding.", "stack": "Traceback (most recent call last):\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py\", line 187, in run\n    self.tearDown()\n  File \"/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_crash.py\", line 87, in tearDown\n    super(BaseCrashTestCase, self).tearDown()\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py\", line 322, in tearDown\n    self.marionette.start_session()\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py\", line 28, in _\n    m._handle_socket_failure()\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py\", line 818, in _handle_socket_failure\n    self.cleanup()\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py\", line 655, in cleanup\n    self.instance.close(clean=True)\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/geckoinstance.py\", line 407, in close\n    \"tcp:{}\".format(self.marionette_port))\n  File \"/builds/worker/workspace/build/venv/lib/python2.7/site-packages/mozdevice/devicemanagerADB.py\", line 211, in remove_forward\n    raise DMError(\"Failed to remove connection forwarding.\")\n"}
This test should not be run on Android. Maybe the skip logic is wrong. I will have a look at it tomorrow.
Flags: needinfo?(hskupin)
I think the test for fennec is okay, but there's just a single '|' earlier in the annotation.
Attachment #8922366 - Flags: review?(hskupin)
Comment on attachment 8922366 [details] [diff] [review]
fix test annotation

Review of attachment 8922366 [details] [diff] [review]:
-----------------------------------------------------------------

Ups! Sorry for that mistake. Thanks for fixing it.
Attachment #8922366 - Flags: review?(hskupin) → review+
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/77cbe7b85ee2
Fix manifest annotation for test_crash.py; r=whimboo
https://hg.mozilla.org/mozilla-central/rev/77cbe7b85ee2
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox58: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla58
Whiteboard: [stockwell disabled]
Joel, this test is not disabled per-se. But only for those platforms it cannot be run on. As such I remove your whiteboard entry again.
Flags: needinfo?(hskupin)
Whiteboard: [stockwell disabled]
:whimboo, how would you categorize the fix?  If this doesn't run on Android, we disabled the test from running there to fix the problem, correct?  I do not see a fix to the product or a fix to the test in order to see this resolved.
Flags: needinfo?(hskupin)
This test cannot be run on Android because the application is not controlled by Marionette. The fix here was only a regression from by patch on bug 1410355, and didn't change anything in regards of where the tests are getting run.
Flags: needinfo?(hskupin)
given this the problem of a perma failing test was resolved by disabling it on the platform where it was perma failing.  We have marked many tests as disabled for turning off jobs where they are not wanted anymore, etc.
Whiteboard: [stockwell disabled]
Assignee: nobody → gbrown
Product: Testing → Remote Protocol
Moving bug to Testing::Marionette Client and Harness component per bug 1815831.
Component: Marionette → Marionette Client and Harness
Product: Remote Protocol → Testing
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: