Closed
Bug 1331455
Opened 7 years ago
Closed 7 years ago
Intermittent Android OSError: [Errno 2] No such file or directory - unable to run netstat
Categories
(Firefox for Android Graveyard :: Testing, defect, P1)
Firefox for Android Graveyard
Testing
Tracking
(firefox50 unaffected, firefox51 unaffected, firefox52 unaffected, firefox53 fixed)
RESOLVED
FIXED
Firefox 53
Tracking | Status | |
---|---|---|
firefox50 | --- | unaffected |
firefox51 | --- | unaffected |
firefox52 | --- | unaffected |
firefox53 | --- | fixed |
People
(Reporter: gbrown, Assigned: gbrown)
References
Details
(Keywords: intermittent-failure)
Attachments
(1 file)
687 bytes,
patch
|
jmaher
:
review+
|
Details | Diff | Splinter Review |
+++ This bug was initially created as a clone of Bug #1324210 +++ Several Android intermittent failures are recorded in bug 1324210, beginning Jan 12, caused by a failure to run netstat. I think this was regressed by bug 1330408, which switched some Android jobs to a Ubuntu 16 docker image.
Assignee | ||
Comment 1•7 years ago
|
||
https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=69364108&lineNumber=1499 [task 2017-01-16T19:55:25.314223Z] 19:55:25 INFO - Running netstat -a -p -n -t -u [task 2017-01-16T19:55:25.359580Z] 19:55:25 INFO - Running post-action listener: _resource_record_post_action [task 2017-01-16T19:55:25.359922Z] 19:55:25 INFO - [mozharness: 2017-01-16 19:55:25.359765Z] Finished verify-emulator step (failed) [task 2017-01-16T19:55:25.361500Z] 19:55:25 FATAL - Uncaught exception: Traceback (most recent call last): [task 2017-01-16T19:55:25.361937Z] 19:55:25 FATAL - File "/home/worker/workspace/mozharness/mozharness/base/script.py", line 2028, in run [task 2017-01-16T19:55:25.362362Z] 19:55:25 FATAL - self.run_action(action) [task 2017-01-16T19:55:25.362965Z] 19:55:25 FATAL - File "/home/worker/workspace/mozharness/mozharness/base/script.py", line 1967, in run_action [task 2017-01-16T19:55:25.363220Z] 19:55:25 FATAL - self._possibly_run_method(method_name, error_if_missing=True) [task 2017-01-16T19:55:25.363662Z] 19:55:25 FATAL - File "/home/worker/workspace/mozharness/mozharness/base/script.py", line 1907, in _possibly_run_method [task 2017-01-16T19:55:25.364095Z] 19:55:25 FATAL - return getattr(self, method_name)() [task 2017-01-16T19:55:25.364539Z] 19:55:25 FATAL - File "/home/worker/workspace/mozharness/scripts/android_emulator_unittest.py", line 627, in verify_emulator [task 2017-01-16T19:55:25.364974Z] 19:55:25 FATAL - emulator_ok = self._retry(max_restarts, 10, self._verify_emulator_and_restart_on_fail, "Check emulator") [task 2017-01-16T19:55:25.365404Z] 19:55:25 FATAL - File "/home/worker/workspace/mozharness/scripts/android_emulator_unittest.py", line 244, in _retry [task 2017-01-16T19:55:25.365895Z] 19:55:25 FATAL - status = func() [task 2017-01-16T19:55:25.366309Z] 19:55:25 FATAL - File "/home/worker/workspace/mozharness/scripts/android_emulator_unittest.py", line 339, in _verify_emulator_and_restart_on_fail [task 2017-01-16T19:55:25.366701Z] 19:55:25 FATAL - self._dump_host_state() [task 2017-01-16T19:55:25.367231Z] 19:55:25 FATAL - File "/home/worker/workspace/mozharness/scripts/android_emulator_unittest.py", line 376, in _dump_host_state [task 2017-01-16T19:55:25.367550Z] 19:55:25 FATAL - self._run_proc(['netstat', '-a', '-p', '-n', '-t', '-u']) [task 2017-01-16T19:55:25.368049Z] 19:55:25 FATAL - File "/home/worker/workspace/mozharness/scripts/android_emulator_unittest.py", line 253, in _run_proc [task 2017-01-16T19:55:25.368440Z] 19:55:25 FATAL - p = subprocess.Popen(cmd, stdout=subprocess.PIPE) [task 2017-01-16T19:55:25.368835Z] 19:55:25 FATAL - File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ [task 2017-01-16T19:55:25.369327Z] 19:55:25 FATAL - errread, errwrite) [task 2017-01-16T19:55:25.369731Z] 19:55:25 FATAL - File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child [task 2017-01-16T19:55:25.370176Z] 19:55:25 FATAL - raise child_exception [task 2017-01-16T19:55:25.370764Z] 19:55:25 FATAL - OSError: [Errno 2] No such file or directory [task 2017-01-16T19:55:25.371276Z] 19:55:25 FATAL - Running post_fatal callback... [task 2017-01-16T19:55:25.371776Z] 19:55:25 FATAL - Exiting -1 [task 2017-01-16T19:55:25.372383Z] 19:55:25 INFO - Running post-run listener: _resource_record_post_run
Assignee | ||
Comment 2•7 years ago
|
||
android_emulator_unittest.py normally does not call netstat. It only calls netstat when the emulator fails to start up. Normally, when that happens, mozharness dumps additional info - such as netstat output - and then retries the job automatically; those retries are now being reported as netstat failures. I guess netstat is simply missing from the Ubuntu 16 docker image.
Assignee | ||
Comment 3•7 years ago
|
||
Verified on a loaner that netstat is not installed, and can be: root@taskcluster-worker:~# netstat bash: netstat: command not found root@taskcluster-worker:~# apt install net-tools ... root@taskcluster-worker:~# netstat -a -p -n -t -u Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 172.17.0.5:44855 91.189.88.162:80 TIME_WAIT - root@taskcluster-worker:~#
Assignee | ||
Comment 4•7 years ago
|
||
Add net-tools to ubuntu 16.04 docker image, for netstat command, used by Android test jobs. No harm done: https://treeherder.mozilla.org/#/jobs?repo=try&revision=412b475b80a2a2879afd11ff87bb0feeb9d730da
Attachment #8827285 -
Flags: review?(jmaher)
Assignee | ||
Updated•7 years ago
|
Priority: -- → P1
Updated•7 years ago
|
Attachment #8827285 -
Flags: review?(jmaher) → review+
Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/0159194557cf Add net-tools to Ubuntu 16 docker image, for netstat; r=jmaher
Comment 6•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/0159194557cf
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox53:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 53
Updated•7 years ago
|
status-firefox50:
--- → unaffected
status-firefox51:
--- → unaffected
status-firefox52:
--- → unaffected
Comment hidden (Intermittent Failures Robot) |
Updated•3 years ago
|
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•