1128745 - Timeout during Android xpcshell test intermittently reports "AttributeError: 'RemoteXPCShellTestThread' object has no attribute 'shellReturnCode'"

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Geoff Brown [:gbrown]]

As seen in bug 971742 and several other intermittent android xpcshell test failures, in the event of a test timeout, logs sometimes show:

AttributeError: 'RemoteXPCShellTestThread' object has no attribute 'shellReturnCode'

Comment 1

[Geoff Brown [:gbrown]]

http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/fx-team-android-api-9/1422452016/fx-team_ubuntu64_vm_mobile_test-xpcshell-1-bm67-tests1-linux64-build110.txt.gz

09:28:10     INFO -  TEST-START | netwerk/test/unit/test_URIs.js
09:28:10  WARNING -  TEST-UNEXPECTED-TIMEOUT | netwerk/test/unit/test_URIs.js | Test timed out
09:28:10     INFO -  TEST-INFO took 300001ms
09:28:10     INFO -  mozdevice unable to kill -1 xpcshell -- not running?
09:28:10     INFO -  Exception in thread Thread-1094:
09:28:10     INFO -  Traceback (most recent call last):
09:28:10     INFO -    File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
09:28:10     INFO -      self.run()
09:28:10     INFO -    File "/usr/lib/python2.7/threading.py", line 755, in run
09:28:10     INFO -      self.function(*self.args, **self.kwargs)
09:28:10     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 642, in <lambda>
09:28:10     INFO -      testTimer = Timer(testTimeoutInterval, lambda: self.testTimeout(proc))
09:28:10     INFO -    File "/builds/slave/test/build/tests/xpcshell/runxpcshelltests.py", line 304, in testTimeout
09:28:10     INFO -      self.log.info("xpcshell return code: %s" % self.getReturnCode(proc))
09:28:10     INFO -    File "/builds/slave/test/build/tests/xpcshell/remotexpcshelltests.py", line 186, in getReturnCode
09:28:10     INFO -      if self.shellReturnCode is not None:
09:28:10     INFO -  AttributeError: 'RemoteXPCShellTestThread' object has no attribute 'shellReturnCode'
09:28:10     INFO -  
09:28:10  WARNING -  TEST-UNEXPECTED-FAIL | Received SIGINT (control-C), so stopped run. (Use --keep-going to keep running tests after killing one with SIGINT)
09:28:10     INFO -  INFO | Result summary:
09:28:10     INFO -  INFO | Passed: 446
09:28:10  WARNING -  INFO | Failed: 1
09:28:10  WARNING -  One or more unittests failed.
09:28:10     INFO -  INFO | Todo: 4
09:28:10     INFO -  INFO | Retried: 0
09:28:10     INFO -  SUITE-END | took 6324s

Comment 2

[Geoff Brown [:gbrown]]

Counter example:

http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/fx-team-android-api-9/1422879935/fx-team_ubuntu64_vm_mobile_test-xpcshell-1-bm67-tests1-linux64-build132.txt.gz

07:59:54     INFO -  TEST-START | netwerk/test/unit/test_URIs.js
07:59:54  WARNING -  TEST-UNEXPECTED-TIMEOUT | netwerk/test/unit/test_URIs.js | Test timed out
07:59:54     INFO -  TEST-INFO took 300001ms
07:59:54     INFO -  mozdevice unable to kill -1 xpcshell -- not running?
07:59:54     INFO -  xpcshell return code: 0
07:59:54  WARNING -  TEST-UNEXPECTED-FAIL | Received SIGINT (control-C), so stopped run. (Use --keep-going to keep running tests after killing one with SIGINT)
07:59:54     INFO -  INFO | Result summary:
07:59:54     INFO -  INFO | Passed: 446
07:59:54  WARNING -  INFO | Failed: 1
07:59:54  WARNING -  One or more unittests failed.
07:59:54     INFO -  INFO | Todo: 4
07:59:54     INFO -  INFO | Retried: 0
07:59:54     INFO -  SUITE-END | took 6071s

Comment 3

[Geoff Brown [:gbrown]]

Attachment: make sure shellReturnCode is defined

I think this is straight-forward and trivial to fix.

A RemoteXPCShellTestThread instance is created for each test. RemoteXPCShellTestThread.shellReturnCode is defined when launchProcess() returns. But if there is a timeout before launchProcess returns, getReturnCode will try to access shellReturnCode before it is defined.

Let's just define shellReturnCode when RemoteXPCShellTestThread is created; getReturnCode already handles a shellReturnCode of None, so as long as the attribute exists, normal timeout handling should be able to proceed.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8561421 [details] [diff] [review]
make sure shellReturnCode is defined

Review of attachment 8561421 [details] [diff] [review]:
-----------------------------------------------------------------

thanks!

Comment 5

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/8b3f729e1322

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/8b3f729e1322