1555563 - Intermittent Return code: -4 after {'VIRTUALENV_NO_DOWNLOAD': '1'} for macosx tests

Status: RESOLVED INCOMPLETE

(Testing :: Mochitest, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: cbrindusan [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=249088626&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/LLH7TbV6SeWs_cnjulzSWA/runs/0/artifacts/public/logs/live_backing.log

---

01:08:29 INFO - [mozharness: 2019-05-30 01:08:29.462248Z] Finished download-and-extract step (success)
01:08:29 INFO - [mozharness: 2019-05-30 01:08:29.462346Z] Running create-virtualenv step.
01:08:29 INFO - Running pre-action listener: _install_mozbase
01:08:29 INFO - Running pre-action listener: _pre_create_virtualenv
01:08:29 INFO - Running pre-action listener: _resource_record_pre_action
01:08:29 INFO - Running main action method: create_virtualenv
01:08:29 INFO - Creating virtualenv /Users/cltbld/tasks/task_1559178445/build/venv
01:08:29 INFO - Running command: ['/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', '/Users/cltbld/tasks/task_1559178445/mozharness/external_tools/virtualenv/virtualenv.py', '--always-copy', '/Users/cltbld/tasks/task_1559178445/build/venv'] in /Users/cltbld/tasks/task_1559178445/build
01:08:29 INFO - Copy/paste: /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /Users/cltbld/tasks/task_1559178445/mozharness/external_tools/virtualenv/virtualenv.py --always-copy /Users/cltbld/tasks/task_1559178445/build/venv
01:08:29 INFO - Using partial env: {'VIRTUALENV_NO_DOWNLOAD': '1'}
01:08:29 ERROR - Return code: -4
01:08:29 ERROR - -4 not in success codes: [0]
01:08:29 WARNING - setting return code to 2
01:08:29 FATAL - Halting on failure while running ['/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', '/Users/cltbld/tasks/task_1559178445/mozharness/external_tools/virtualenv/virtualenv.py', '--always-copy', '/Users/cltbld/tasks/task_1559178445/build/venv']
01:08:29 FATAL - Running post_fatal callback...
01:08:29 FATAL - Exiting 2
01:08:29 INFO - Running post-action listener: _resource_record_post_action
01:08:29 INFO - Running post-action listener: _start_resource_monitoring
01:08:29 ERROR - Exception during post-action for create-virtualenv: Traceback (most recent call last):
01:08:29 ERROR - File "/Users/cltbld/tasks/task_1559178445/mozharness/mozharness/base/script.py", line 2048, in run_action
01:08:29 ERROR - method(action, success=success and self.return_code == 0)
01:08:29 ERROR - File "/Users/cltbld/tasks/task_1559178445/mozharness/mozharness/base/python.py", line 514, in _start_resource_monitoring
01:08:29 ERROR - self.activate_virtualenv()
01:08:29 ERROR - File "/Users/cltbld/tasks/task_1559178445/mozharness/mozharness/base/python.py", line 438, in activate_virtualenv
01:08:29 ERROR - execfile(activate, dict(file=activate))
01:08:29 ERROR - IOError: [Errno 2] No such file or directory: '/Users/cltbld/tasks/task_1559178445/build/venv/bin/activate_this.py'
01:08:29 INFO - [mozharness: 2019-05-30 01:08:29.748918Z] Finished create-virtualenv step (failed)
01:08:29 FATAL - Aborting due to failure in post-action listener.
01:08:29 FATAL - Running post_fatal callback...
01:08:29 FATAL - Exiting -1
01:08:29 INFO - Running post-run listener: _resource_record_post_run
[taskcluster 2019-05-30T01:08:29.799Z] Exit Code: 255
[taskcluster 2019-05-30T01:08:29.799Z] User Time: 32.290943s
[taskcluster 2019-05-30T01:08:29.799Z] Kernel Time: 16.80145s
[taskcluster 2019-05-30T01:08:29.799Z] Wall Time: 57.370123243s
[taskcluster 2019-05-30T01:08:29.799Z] Result: FAILED
[taskcluster 2019-05-30T01:08:29.799Z] === Task Finished ===
[taskcluster 2019-05-30T01:08:29.799Z] Task Duration: 57.370353994s
[taskcluster 2019-05-30T01:08:30.342Z] Uploading artifact public/logs/localconfig.json from file logs/localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2020-05-29T00:36:13.320Z
[taskcluster:error] Uploading error artifact public/test_info from file build/blobber_upload_dir with message "Could not read directory '/Users/cltbld/tasks/task_1559178445/build/blobber_upload_dir'", reason "file-missing-on-worker" and expiry 2020-05-29T00:36:13.320Z
[taskcluster:error] TASK FAILURE during artifact upload: file-missing-on-worker: Could not read directory '/Users/cltbld/tasks/task_1559178445/build/blobber_upload_dir'
[taskcluster:error] exit status 255
[taskcluster:error] file-missing-on-worker: Could not read directory '/Users/cltbld/tasks/task_1559178445/build/blobber_upload_dir'

Comment 1

[Intermittent Failures Robot]

19 failures in 917 pushes (0.021 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 11
• mozilla-inbound: 2
• mozilla-central: 6

Platform breakdown:

• macosx1010-64-shippable: 11
• macosx1010-64: 7
• macosx1010-64-shippable-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1555563&startday=2019-05-29&endday=2019-05-29&tree=all

Comment 2

[Bogdan Tara[:bogdan_tara | bogdant]]

Started some r/b on range: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=macosx1010-64%2Cdebug%2Cmochitests%2Ctest-macosx1010-64%2Fdebug-mochitest-devtools-chrome-e10s-4%2Cm%28dt4%29&tochange=28b379c201867f1c0f0dc7d05346a25a1012d63d&fromchange=cab2c0eaec747ce6b37df4a963f0a1ff09aca22e&selectedJob=248996988

Comment 3

[Intermittent Failures Robot]

27 failures in 4337 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 19
• mozilla-inbound: 2
• mozilla-central: 6

Platform breakdown:

• macosx1010-64-shippable: 15
• macosx1010-64: 10
• macosx1010-64-shippable-qr: 1
• macosx1010-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1555563&startday=2019-05-27&endday=2019-06-02&tree=all

Comment 4

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup