1593682 - Intermittent ChunkedEncodingError: ('Connection broken: IncompleteRead(1375 bytes read, 6632 more expected)', IncompleteRead(1375 bytes read, 6632 more expected))

Status: RESOLVED INCOMPLETE

(Release Engineering :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ccoroiu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=274384654&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/DIYy6W5mTSe6tW9M09jH_A/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-11-04T13:40:53.589Z] 13:40:53 INFO - File "z:\task_1572872266\build\venv\lib\site-packages\pip_vendor\requests\models.py", line 667, in generate
[task 2019-11-04T13:40:53.589Z] 13:40:53 INFO - raise ChunkedEncodingError(e)
[task 2019-11-04T13:40:53.589Z] 13:40:53 INFO - ChunkedEncodingError: ('Connection broken: IncompleteRead(1375 bytes read, 6632 more expected)', IncompleteRead(1375 bytes read, 6632 more expected))
[task 2019-11-04T13:40:53.606Z] 13:40:53 WARNING - Return code: 2
[task 2019-11-04T13:40:53.607Z] 13:40:53 WARNING - Could not install python package: failed all attempts.
[task 2019-11-04T13:40:53.607Z] 13:40:53 INFO - Installing jsonschema==2.5.1 into virtualenv Z:\task_1572872266\build\venv
[task 2019-11-04T13:40:53.616Z] 13:40:53 INFO - retry: Calling run_command with args: [['Z:\task_1572872266\build\venv\Scripts\pip', 'install', '--timeout', '120', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub', 'jsonschema==2.5.1']], kwargs: {'error_level': 'warning', 'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002FD61F0>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002FEE4E0>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002E09DD0>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002B30BB8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000003078030>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002B839F0>, 'level': 'critical'}], 'cwd': 'Z:\task_1572872266\build', 'env': {'TMP': 'C:\Users\task_1572872266\AppData\Local\Temp', 'USERNAME': 'task_1572872266', 'COMPUTERNAME': 'I-0E93EFAB56A3D', 'USERDOMAIN': 'I-0E93EFAB56A3D', 'MOZ_AUTOMATION': '1', 'PSMODULEPATH': '%ProgramFiles%\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules', 'COMMONPROGRAMFILES': 'C:\Program Files\Common Files', 'TASKCLUSTER_ROOT_URL': 'https://taskcluster.net', 'PROCESSOR_IDENTIFIER': 'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel', 'PROGRAMFILES': 'C:\Program Files', 'PROCESSOR_REVISION': '5504', 'SYSTEMROOT': 'C:\Windows', 'PATH': 'C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Windows\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;;C:\Program Files\Mercurial;C:\mozilla-build\bin;C:\mozilla-build\kdiff3;C:\mozilla-build\moztools-x64\bin;C:\mozilla-build\mozmake;C:\mozilla-build\msys\bin;C:\mozilla-build\msys\local\bin;C:\mozilla-build\nsis-3.01;C:\mozilla-build\python;C:\mozilla-build\python\Scripts;C:\mozilla-build\python3;C:\Users\task_1572872266\AppData\Local\Microsoft\WindowsApps', 'USERPROFILE': 'C:\Users\task_1572872266', 'PROGRAMFILES(X86)': 'C:\Program Files (x86)', 'DRIVERDATA': 'C:\Windows\System32\Drivers\DriverData', 'TEMP': 'C:\Users\task_1572872266\AppData\Local\Temp', 'RUN_ID': '0', 'COMMONPROGRAMFILES(X86)': 'C:\Program Files (x86)\Common Files', 'TASKCLUSTER_INSTANCE_TYPE': 'c5.2xlarge', 'PROCESSOR_ARCHITECTURE': 'AMD64', 'ALLUSERSPROFILE': 'C:\ProgramData', 'EXTRA_MOZHARNESS_CONFIG': '{"test_packages_url": "https://queue.taskcluster.net/v1/task/HgL1XWEwRr2rpEROyaesGA/artifacts/public/build/target.test_packages.json", "installer_url": "https://queue.taskcluster.net/v1/task/HgL1XWEwRr2rpEROyaesGA/artifacts/public/build/target.zip"}', 'LOCALAPPDATA': 'C:\Users\task_1572872266\AppData\Local', 'HOMEPATH': '\Users\task_1572872266', 'USERDOMAIN_ROAMINGPROFILE': 'I-0E93EFAB56A3D', 'PROGRAMW6432': 'C:\Program Files', 'MOZILLABUILD': 'C:\mozilla-build', 'SCCACHE_DISABLE': '1', 'LOGONSERVER': '\\I-0E93EFAB56A3D', 'PROMPT': '$P$G', 'COMSPEC': 'C:\Windows\system32\cmd.exe', 'PROGRAMDATA': 'C:\ProgramData', 'GECKO_HEAD_REV': '81e027b9973731be9522e96d48a2b0b0ecbcf0b4', 'ONEDRIVE': 'C:\Users\task_1572872266\OneDrive', 'MOZ_SCM_LEVEL': '3', 'GECKO_HEAD_REPOSITORY': 'https://hg.mozilla.org/integration/autoland', 'TASKCLUSTER_WORKER_LOCATION': '{"cloud":"aws","region":"eu-central-1","availabilityZone":"eu-central-1b"}', 'PATHEXT': '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC', 'WINDIR': 'C:\Windows', 'APPDATA': 'Z:\task_1572872266\AppData\Roaming', 'HOMEDRIVE': 'C:', 'SYSTEMDRIVE': 'C:', 'NUMBER_OF_PROCESSORS': '8', 'TASK_ID': 'DIYy6W5mTSe6tW9M09jH_A', 'PIP_DOWNLOAD_CACHE': 'y:\pip-cache', 'PROCESSOR_LEVEL': '6', 'COMMONPROGRAMW6432': 'C:\Program Files\Common Files', 'OS': 'Windows_NT', 'PUBLIC': 'C:\Users\Public', 'TOOLTOOL_CACHE': 'y:\tooltool-cache'}}, attempt #1
[task 2019-11-04T13:40:53.616Z] 13:40:53 INFO - Running command: ['Z:\task_1572872266\build\venv\Scripts\pip', 'install', '--timeout', '120', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub', 'jsonschema==2.5.1'] in Z:\task_1572872266\build
[task 2019-11-04T13:40:53.616Z] 13:40:53 INFO - Copy/paste: Z:\task_1572872266\build\venv\Scripts\pip install --timeout 120 --no-index --find-links https://pypi.pub.build.mozilla.org/pub jsonschema==2.5.1
[task 2019-11-04T13:40:53.616Z] 13:40:53 INFO - Using env: (same as previous command)
[task 2019-11-04T13:40:53.877Z] 13:40:53 INFO - Ignoring indexes: https://pypi.python.org/simple
[task 2019-11-04T13:40:53.885Z] 13:40:53 INFO - Collecting jsonschema==2.5.1
[task 2019-11-04T13:46:56.078Z] 13:46:56 INFO - Could not find a version that satisfies the requirement jsonschema==2.5.1 (from versions: )
[task 2019-11-04T13:46:56.079Z] 13:46:56 INFO - No matching distribution found for jsonschema==2.5.1
[task 2019-11-04T13:46:56.101Z] 13:46:56 WARNING - Return code: 1
[task 2019-11-04T13:46:56.101Z] 13:46:56 INFO - retry: Failed, sleeping 60 seconds before retrying
[task 2019-11-04T13:47:56.103Z] 13:47:56 INFO - retry: Calling run_command with args: [['Z:\task_1572872266\build\venv\Scripts\pip', 'install', '--timeout', '120', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub', 'jsonschema==2.5.1']], kwargs: {'error_level': 'warning', 'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002FD61F0>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002FEE4E0>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002E09DD0>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002B30BB8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000003078030>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002B839F0>, 'level': 'critical'}], 'cwd': 'Z:\task_1572872266\build', 'env': {'TMP': 'C:\Users\task_1572872266\AppData\Local\Temp', 'USERNAME': 'task_1572872266', 'COMPUTERNAME': 'I-0E93EFAB56A3D', 'USERDOMAIN': 'I-0E93EFAB56A3D', 'MOZ_AUTOMATION': '1', 'PSMODULEPATH': '%ProgramFiles%\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules', 'COMMONPROGRAMFILES': 'C:\Program Files\Common Files', 'TASKCLUSTER_ROOT_URL': 'https://taskcluster.net', 'PROCESSOR_IDENTIFIER': 'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel', 'PROGRAMFILES': 'C:\Program Files', 'PROCESSOR_REVISION': '5504', 'SYSTEMROOT': 'C:\Windows', 'PATH': 'C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Windows\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;;C:\Program Files\Mercurial;C:\mozilla-build\bin;C:\mozilla-build\kdiff3;C:\mozilla-build\moztools-x64\bin;C:\mozilla-build\mozmake;C:\mozilla-build\msys\bin;C:\mozilla-build\msys\local\bin;C:\mozilla-build\nsis-3.01;C:\mozilla-build\python;C:\mozilla-build\python\Scripts;C:\mozilla-build\python3;C:\Users\task_1572872266\AppData\Local\Microsoft\WindowsApps', 'USERPROFILE': 'C:\Users\task_1572872266', 'PROGRAMFILES(X86)': 'C:\Program Files (x86)', 'DRIVERDATA': 'C:\Windows\System32\Drivers\DriverData', 'TEMP': 'C:\Users\task_1572872266\AppData\Local\Temp', 'RUN_ID': '0', 'COMMONPROGRAMFILES(X86)': 'C:\Program Files (x86)\Common Files', 'TASKCLUSTER_INSTANCE_TYPE': 'c5.2xlarge', 'PROCESSOR_ARCHITECTURE': 'AMD64', 'ALLUSERSPROFILE': 'C:\ProgramData', 'EXTRA_MOZHARNESS_CONFIG': '{"test_packages_url": "https://queue.taskcluster.net/v1/task/HgL1XWEwRr2rpEROyaesGA/artifacts/public/build/target.test_packages.json", "installer_url": "https://queue.taskcluster.net/v1/task/HgL1XWEwRr2rpEROyaesGA/artifacts/public/build/target.zip"}', 'LOCALAPPDATA': 'C:\Users\task_1572872266\AppData\Local', 'HOMEPATH': '\Users\task_1572872266', 'USERDOMAIN_ROAMINGPROFILE': 'I-0E93EFAB56A3D', 'PROGRAMW6432': 'C:\Program Files', 'MOZILLABUILD': 'C:\mozilla-build', 'SCCACHE_DISABLE': '1', 'LOGONSERVER': '\\I-0E93EFAB56A3D', 'PROMPT': '$P$G', 'COMSPEC': 'C:\Windows\system32\cmd.exe', 'PROGRAMDATA': 'C:\ProgramData', 'GECKO_HEAD_REV': '81e027b9973731be9522e96d48a2b0b0ecbcf0b4', 'ONEDRIVE': 'C:\Users\task_1572872266\OneDrive', 'MOZ_SCM_LEVEL': '3', 'GECKO_HEAD_REPOSITORY': 'https://hg.mozilla.org/integration/autoland', 'TASKCLUSTER_WORKER_LOCATION': '{"cloud":"aws","region":"eu-central-1","availabilityZone":"eu-central-1b"}', 'PATHEXT': '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC', 'WINDIR': 'C:\Windows', 'APPDATA': 'Z:\task_1572872266\AppData\Roaming', 'HOMEDRIVE': 'C:', 'SYSTEMDRIVE': 'C:', 'NUMBER_OF_PROCESSORS': '8', 'TASK_ID': 'DIYy6W5mTSe6tW9M09jH_A', 'PIP_DOWNLOAD_CACHE': 'y:\pip-cache', 'PROCESSOR_LEVEL': '6', 'COMMONPROGRAMW6432': 'C:\Program Files\Common Files', 'OS': 'Windows_NT', 'PUBLIC': 'C:\Users\Public', 'TOOLTOOL_CACHE': 'y:\tooltool-cache'}}, attempt #2
[task 2019-11-04T13:47:56.103Z] 13:47:56 INFO - Running command: ['Z:\task_1572872266\build\venv\Scripts\pip', 'install', '--timeout', '120', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub', 'jsonschema==2.5.1'] in Z:\task_1572872266\build
[task 2019-11-04T13:47:56.103Z] 13:47:56 INFO - Copy/paste: Z:\task_1572872266\build\venv\Scripts\pip install --timeout 120 --no-index --find-links https://pypi.pub.build.mozilla.org/pub jsonschema==2.5.1
[task 2019-11-04T13:47:56.103Z] 13:47:56 INFO - Using env: (same as previous command)
[task 2019-11-04T13:47:56.413Z] 13:47:56 INFO - Ignoring indexes: https://pypi.python.org/simple
[task 2019-11-04T13:47:56.424Z] 13:47:56 INFO - Collecting jsonschema==2.5.1
[task 2019-11-04T13:47:58.637Z] 13:47:58 INFO - Downloading https://pypi.pub.build.mozilla.org/pub/jsonschema-2.5.1-py2.py3-none-any.whl
[task 2019-11-04T13:47:58.672Z] 13:47:58 INFO - Collecting functools32; python_version == "2.7" (from jsonschema==2.5.1)
[task 2019-11-04T13:47:59.599Z] 13:47:59 INFO - Downloading https://pypi.pub.build.mozilla.org/pub/functools32-3.2.3-2.tar.gz
[task 2019-11-04T13:47:59.865Z] 13:47:59 INFO - Building wheels for collected packages: functools32
[task 2019-11-04T13:47:59.867Z] 13:47:59 INFO - Running setup.py bdist_wheel for functools32: started
[task 2019-11-04T13:48:00.149Z] 13:48:00 INFO - Running setup.py bdist_wheel for functools32: finished with status 'done'
[task 2019-11-04T13:48:00.150Z] 13:48:00 INFO - Stored in directory: C:\Users\task_1572872266\AppData\Local\pip\Cache\wheels\58\14\4c\707c05ed77f528c717db124c374fa788560a0cb60491a6d1f8
[task 2019-11-04T13:48:00.269Z] 13:48:00 INFO - Successfully built functools32
[task 2019-11-04T13:48:00.269Z] 13:48:00 INFO - Installing collected packages: functools32, jsonschema
[task 2019-11-04T13:48:00.417Z] 13:48:00 INFO - Successfully installed functools32-3.2.3.post2 jsonschema-2.5.1
[task 2019-11-04T13:48:00.470Z] 13:48:00 INFO - Return code: 0
[task 2019-11-04T13:48:00.470Z] 13:48:00 INFO - Installing functools32==3.2.3-2 into virtualenv Z:\task_1572872266\build\venv
[task 2019-11-04T13:48:00.473Z] 13:48:00 INFO - retry: Calling run_command with args: [['Z:\task_1572872266\build\venv\Scripts\pip', 'install', '--timeout', '120', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub', 'functools32==3.2.3-2']], kwargs: {'error_level': 'warning', 'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002FD61F0>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002FEE4E0>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002E09DD0>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002B30BB8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000003078030>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002B839F0>, 'level': 'critical'}], 'cwd': 'Z:\task_1572872266\build', 'env': {'TMP': 'C:\Users\task_1572872266\AppData\Local\Temp', 'USERNAME': 'task_1572872266', 'COMPUTERNAME': 'I-0E93EFAB56A3D', 'USERDOMAIN': 'I-0E93EFAB56A3D', 'MOZ_AUTOMATION': '1', 'PSMODULEPATH': '%ProgramFiles%\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules', 'COMMONPROGRAMFILES': 'C:\Program Files\Common Files', 'TASKCLUSTER_ROOT_URL': 'https://taskcluster.net', 'PROCESSOR_IDENTIFIER': 'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel', 'PROGRAMFILES': 'C:\Program Files', 'PROCESSOR_REVISION': '5504', 'SYSTEMROOT': 'C:\Windows', 'PATH': 'C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Windows\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;;C:\Program Files\Mercurial;C:\mozilla-build\bin;C:\mozilla-build\kdiff3;C:\mozilla-build\moztools-x64\bin;C:\mozilla-build\mozmake;C:\mozilla-build\msys\bin;C:\mozilla-build\msys\local\bin;C:\mozilla-build\nsis-3.01;C:\mozilla-build\python;C:\mozilla-build\python\Scripts;C:\mozilla-build\python3;C:\Users\task_1572872266\AppData\Local\Microsoft\WindowsApps', 'USERPROFILE': 'C:\Users\task_1572872266', 'PROGRAMFILES(X86)': 'C:\Program Files (x86)', 'DRIVERDATA': 'C:\Windows\System32\Drivers\DriverData', 'TEMP': 'C:\Users\task_1572872266\AppData\Local\Temp', 'RUN_ID': '0', 'COMMONPROGRAMFILES(X86)': 'C:\Program Files (x86)\Common Files', 'TASKCLUSTER_INSTANCE_TYPE': 'c5.2xlarge', 'PROCESSOR_ARCHITECTURE': 'AMD64', 'ALLUSERSPROFILE': 'C:\ProgramData', 'EXTRA_MOZHARNESS_CONFIG': '{"test_packages_url": "https://queue.taskcluster.net/v1/task/HgL1XWEwRr2rpEROyaesGA/artifacts/public/build/target.test_packages.json", "installer_url": "https://queue.taskcluster.net/v1/task/HgL1XWEwRr2rpEROyaesGA/artifacts/public/build/target.zip"}', 'LOCALAPPDATA': 'C:\Users\task_1572872266\AppData\Local', 'HOMEPATH': '\Users\task_1572872266', 'USERDOMAIN_ROAMINGPROFILE': 'I-0E93EFAB56A3D', 'PROGRAMW6432': 'C:\Program Files', 'MOZILLABUILD': 'C:\mozilla-build', 'SCCACHE_DISABLE': '1', 'LOGONSERVER': '\\I-0E93EFAB56A3D', 'PROMPT': '$P$G', 'COMSPEC': 'C:\Windows\system32\cmd.exe', 'PROGRAMDATA': 'C:\ProgramData', 'GECKO_HEAD_REV': '81e027b9973731be9522e96d48a2b0b0ecbcf0b4', 'ONEDRIVE': 'C:\Users\task_1572872266\OneDrive', 'MOZ_SCM_LEVEL': '3', 'GECKO_HEAD_REPOSITORY': 'https://hg.mozilla.org/integration/autoland', 'TASKCLUSTER_WORKER_LOCATION': '{"cloud":"aws","region":"eu-central-1","availabilityZone":"eu-central-1b"}', 'PATHEXT': '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC', 'WINDIR': 'C:\Windows', 'APPDATA': 'Z:\task_1572872266\AppData\Roaming', 'HOMEDRIVE': 'C:', 'SYSTEMDRIVE': 'C:', 'NUMBER_OF_PROCESSORS': '8', 'TASK_ID': 'DIYy6W5mTSe6tW9M09jH_A', 'PIP_DOWNLOAD_CACHE': 'y:\pip-cache', 'PROCESSOR_LEVEL': '6', 'COMMONPROGRAMW6432': 'C:\Program Files\Common Files', 'OS': 'Windows_NT', 'PUBLIC': 'C:\Users\Public', 'TOOLTOOL_CACHE': 'y:\tooltool-cache'}}, attempt #1
[task 2019-11-04T13:48:00.473Z] 13:48:00 INFO - Running command: ['Z:\task_1572872266\build\venv\Scripts\pip', 'install', '--timeout', '120', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub', 'functools32==3.2.3-2'] in Z:\task_1572872266\build
[task 2019-11-04T13:48:00.473Z] 13:48:00 INFO - Copy/paste: Z:\task_1572872266\build\venv\Scripts\pip install --timeout 120 --no-index --find-links https://pypi.pub.build.mozilla.org/pub functools32==3.2.3-2
[task 2019-11-04T13:48:00.473Z] 13:48:00 INFO - Using env: (same as previous command)
[task 2019-11-04T13:48:00.748Z] 13:48:00 INFO - Ignoring indexes: https://pypi.python.org/simple
[task 2019-11-04T13:48:00.754Z] 13:48:00 INFO - Requirement already satisfied (use --upgrade to upgrade): functools32==3.2.3-2 in z:\task_1572872266\build\venv\lib\site-packages
[task 2019-11-04T13:48:00.774Z] 13:48:00 INFO - Return code: 0

Comment 1

[Cristina Coroiu [:ccoroiu]]

[task 2019-11-04T14:09:49.006Z] 14:09:49 INFO - File "z:\task_1572872266\build\venv\lib\site-packages\pip_vendor\requests\sessions.py", line 179, in resolve_redirects
[task 2019-11-04T14:09:49.006Z] 14:09:49 INFO - **adapter_kwargs
[task 2019-11-04T14:09:49.006Z] 14:09:49 INFO - File "z:\task_1572872266\build\venv\lib\site-packages\pip_vendor\requests\sessions.py", line 617, in send
[task 2019-11-04T14:09:49.006Z] 14:09:49 INFO - r.content
[task 2019-11-04T14:09:49.006Z] 14:09:49 INFO - File "z:\task_1572872266\build\venv\lib\site-packages\pip_vendor\requests\models.py", line 741, in content
[task 2019-11-04T14:09:49.006Z] 14:09:49 INFO - self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
[task 2019-11-04T14:09:49.006Z] 14:09:49 INFO - File "z:\task_1572872266\build\venv\lib\site-packages\pip_vendor\requests\models.py", line 667, in generate
[task 2019-11-04T14:09:49.007Z] 14:09:49 INFO - raise ChunkedEncodingError(e)
[task 2019-11-04T14:09:49.007Z] 14:09:49 INFO - ChunkedEncodingError: ('Connection broken: IncompleteRead(1058 bytes read, 6961 more expected)', IncompleteRead(1058 bytes read, 6961 more expected))
[task 2019-11-04T14:09:49.025Z] 14:09:49 WARNING - Return code: 2
[task 2019-11-04T14:09:49.025Z] 14:09:49 INFO - retry: Failed, sleeping 300 seconds before retrying
[task 2019-11-04T14:14:49.027Z] 14:14:49 INFO - retry: Calling run_command with args: [['Z:\task_1572872266\build\venv\Scripts\pip', 'install', '--timeout', '120', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub', 'simplejson']], kwargs: {'error_level': 'warning', 'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002FD61F0>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002FEE4E0>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002E09DD0>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002B30BB8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x0000000003078030>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x0000000002B839F0>, 'level': 'critical'}], 'cwd': 'Z:\task_1572872266\build', 'env': {'TMP': 'C:\Users\task_1572872266\AppData\Local\Temp', 'USERNAME': 'task_1572872266', 'COMPUTERNAME': 'I-0E93EFAB56A3D', 'USERDOMAIN': 'I-0E93EFAB56A3D', 'MOZ_AUTOMATION': '1', 'PSMODULEPATH': '%ProgramFiles%\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules', 'COMMONPROGRAMFILES': 'C:\Program Files\Common Files', 'TASKCLUSTER_ROOT_URL': 'https://taskcluster.net', 'PROCESSOR_IDENTIFIER': 'Intel64 Family 6 Model 85 Stepping 4, GenuineIntel', 'PROGRAMFILES': 'C:\Program Files', 'PROCESSOR_REVISION': '5504', 'SYSTEMROOT': 'C:\Windows', 'PATH': 'C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Windows\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;;C:\Program Files\Mercurial;C:\mozilla-build\bin;C:\mozilla-build\kdiff3;C:\mozilla-build\moztools-x64\bin;C:\mozilla-build\mozmake;C:\mozilla-build\msys\bin;C:\mozilla-build\msys\local\bin;C:\mozilla-build\nsis-3.01;C:\mozilla-build\python;C:\mozilla-build\python\Scripts;C:\mozilla-build\python3;C:\Users\task_1572872266\AppData\Local\Microsoft\WindowsApps', 'USERPROFILE': 'C:\Users\task_1572872266', 'PROGRAMFILES(X86)': 'C:\Program Files (x86)', 'DRIVERDATA': 'C:\Windows\System32\Drivers\DriverData', 'TEMP': 'C:\Users\task_1572872266\AppData\Local\Temp', 'RUN_ID': '0', 'COMMONPROGRAMFILES(X86)': 'C:\Program Files (x86)\Common Files', 'TASKCLUSTER_INSTANCE_TYPE': 'c5.2xlarge', 'PROCESSOR_ARCHITECTURE': 'AMD64', 'ALLUSERSPROFILE': 'C:\ProgramData', 'EXTRA_MOZHARNESS_CONFIG': '{"test_packages_url": "https://queue.taskcluster.net/v1/task/HgL1XWEwRr2rpEROyaesGA/artifacts/public/build/target.test_packages.json", "installer_url": "https://queue.taskcluster.net/v1/task/HgL1XWEwRr2rpEROyaesGA/artifacts/public/build/target.zip"}', 'LOCALAPPDATA': 'C:\Users\task_1572872266\AppData\Local', 'HOMEPATH': '\Users\task_1572872266', 'USERDOMAIN_ROAMINGPROFILE': 'I-0E93EFAB56A3D', 'PROGRAMW6432': 'C:\Program Files', 'MOZILLABUILD': 'C:\mozilla-build', 'SCCACHE_DISABLE': '1', 'LOGONSERVER': '\\I-0E93EFAB56A3D', 'PROMPT': '$P$G', 'COMSPEC': 'C:\Windows\system32\cmd.exe', 'PROGRAMDATA': 'C:\ProgramData', 'GECKO_HEAD_REV': '81e027b9973731be9522e96d48a2b0b0ecbcf0b4', 'ONEDRIVE': 'C:\Users\task_1572872266\OneDrive', 'MOZ_SCM_LEVEL': '3', 'GECKO_HEAD_REPOSITORY': 'https://hg.mozilla.org/integration/autoland', 'TASKCLUSTER_WORKER_LOCATION': '{"cloud":"aws","region":"eu-central-1","availabilityZone":"eu-central-1b"}', 'PATHEXT': '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC', 'WINDIR': 'C:\Windows', 'APPDATA': 'Z:\task_1572872266\AppData\Roaming', 'HOMEDRIVE': 'C:', 'SYSTEMDRIVE': 'C:', 'NUMBER_OF_PROCESSORS': '8', 'TASK_ID': 'DIYy6W5mTSe6tW9M09jH_A', 'PIP_DOWNLOAD_CACHE': 'y:\pip-cache', 'PROCESSOR_LEVEL': '6', 'COMMONPROGRAMW6432': 'C:\Program Files\Common Files', 'OS': 'Windows_NT', 'PUBLIC': 'C:\Users\Public', 'TOOLTOOL_CACHE': 'y:\tooltool-cache'}}, attempt #5
[task 2019-11-04T14:14:49.027Z] 14:14:49 INFO - Running command: ['Z:\task_1572872266\build\venv\Scripts\pip', 'install', '--timeout', '120', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub', 'simplejson'] in Z:\task_1572872266\build
[task 2019-11-04T14:14:49.027Z] 14:14:49 INFO - Copy/paste: Z:\task_1572872266\build\venv\Scripts\pip install --timeout 120 --no-index --find-links https://pypi.pub.build.mozilla.org/pub simplejson
[task 2019-11-04T14:14:49.027Z] 14:14:49 INFO - Using env: (same as previous command)
[task 2019-11-04T14:14:49.290Z] 14:14:49 INFO - Ignoring indexes: https://pypi.python.org/simple
[task 2019-11-04T14:14:49.302Z] 14:14:49 INFO - Collecting simplejson
[task 2019-11-04T14:20:33.224Z] 14:20:33 INFO - Could not find a version that satisfies the requirement simplejson (from versions: )
[task 2019-11-04T14:20:33.224Z] 14:20:33 INFO - No matching distribution found for simplejson
[task 2019-11-04T14:20:33.255Z] 14:20:33 WARNING - Return code: 1
[task 2019-11-04T14:20:33.255Z] 14:20:33 FATAL - Could not install python package: failed all attempts.
[task 2019-11-04T14:20:33.255Z] 14:20:33 FATAL - Running post_fatal callback...
[task 2019-11-04T14:20:33.255Z] 14:20:33 FATAL - Exiting -1
[task 2019-11-04T14:20:33.255Z] 14:20:33 INFO - Running post-action listener: _resource_record_post_action
[task 2019-11-04T14:20:33.255Z] 14:20:33 INFO - Running post-action listener: _start_resource_monitoring
[task 2019-11-04T14:20:33.258Z] 14:20:33 WARNING - Unable to start resource monitor: Traceback (most recent call last):
[task 2019-11-04T14:20:33.258Z] 14:20:33 WARNING - File "Z:\task_1572872266\mozharness\mozharness\base\python.py", line 523, in _start_resource_monitoring
[task 2019-11-04T14:20:33.258Z] 14:20:33 WARNING - from mozsystemmonitor.resourcemonitor import SystemResourceMonitor
[task 2019-11-04T14:20:33.258Z] 14:20:33 WARNING - ImportError: No module named mozsystemmonitor.resourcemonitor
[task 2019-11-04T14:20:33.258Z] 14:20:33 INFO - [mozharness: 2019-11-04 14:20:33.258000Z] Finished create-virtualenv step (failed)
[task 2019-11-04T14:20:33.258Z] 14:20:33 INFO - Running post-run listener: _resource_record_post_run
[taskcluster 2019-11-04T14:20:33.295Z] Exit Code: 4294967295
[taskcluster 2019-11-04T14:20:33.295Z] User Time: 0s
[taskcluster 2019-11-04T14:20:33.295Z] Kernel Time: 15.625ms
[taskcluster 2019-11-04T14:20:33.295Z] Wall Time: 45m48.6230792s
[taskcluster 2019-11-04T14:20:33.295Z] Result: FAILED
[taskcluster 2019-11-04T14:20:33.295Z] === Task Finished ===
[taskcluster 2019-11-04T14:20:33.295Z] Task Duration: 45m48.624062s
[taskcluster 2019-11-04T14:20:33.965Z] Uploading artifact public/logs/localconfig.json from file logs\localconfig.json with content encoding "gzip", mime type "application/octet-stream" and expiry 2020-11-03T12:35:50.937Z
[taskcluster:error] Uploading error artifact public/test_info from file build/blobber_upload_dir with message "Could not read directory 'Z:\task_1572872266\build\blobber_upload_dir'", reason "file-missing-on-worker" and expiry 2020-11-03T12:35:50.937Z
[taskcluster:error] TASK FAILURE during artifact upload: file-missing-on-worker: Could not read directory 'Z:\task_1572872266\build\blobber_upload_dir'
[taskcluster 2019-11-04T14:20:35.627Z] Uploading redirect artifact public/logs/live.log to URL https://queue.taskcluster.net/v1/task/DIYy6W5mTSe6tW9M09jH_A/runs/0/artifacts/public/logs/live_backing.log with mime type "text/plain; charset=utf-8" and expiry 2020-11-03T12:35:50.937Z
[taskcluster:error] exit status 4294967295
[taskcluster:error] file-missing-on-worker: Could not read directory 'Z:\task_1572872266\build\blobber_upload_dir'

Comment 2

[Cristina Coroiu [:ccoroiu]]

Treeherder:
https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=d08754485c250df14b1948ad31f03b8f1316b0aa&failure_classification_id=5&tochange=f335193ba437543cb8065285f393408fcbb8e914&selectedJob=274393551

Comment 3

[Alexandru Michis [:malexandru]]

There are also failures of this kind:
ReadTimeoutError: HTTPSConnectionPool(host='pypi.pub.build.mozilla.org', port=443): Read timed out.

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=274339951&repo=autoland&lineNumber=404

Also: INFO - SSLError: TLS/SSL connection has been closed (EOF) (_ssl.c:590)

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=274386589&repo=mozilla-beta&lineNumber=835

Comment 4

[Intermittent Failures Robot]

17 failures in 862 pushes (0.02 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-beta: 2
• try: 9
• autoland: 6

Platform breakdown:

• windows7-32-devedition: 2
• windows7-32: 1
• windows10-64-shippable: 2
• windows10-64-devedition: 5
• windows7-32-shippable: 1
• windows10-64: 5
• windows10-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1593682&startday=2019-11-04&endday=2019-11-04&tree=all

Comment 5

[Intermittent Failures Robot]

17 failures in 4850 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 9
• autoland: 6
• mozilla-beta: 2

Platform breakdown:

• windows7-32: 1
• windows10-64: 5
• windows7-32-devedition: 2
• windows10-64-qr: 1
• windows10-64-shippable: 2
• windows7-32-shippable: 1
• windows10-64-devedition: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1593682&startday=2019-11-04&endday=2019-11-10&tree=all

Comment 6

[Intermittent Failures Robot]

1 failures in 5015 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1

Platform breakdown:

• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1593682&startday=2020-03-30&endday=2020-04-05&tree=all

Comment 7

[BugBot [:suhaib / :marco]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.