Closed Bug 1371156 Opened 8 years ago Closed 3 years ago

Intermittent ConnectionError: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host'))

Categories

(Release Engineering :: Applications: MozharnessCore, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED INCOMPLETE

People

(Reporter: intermittent-bug-filer, Assigned: gbrown)

References

Details

(Keywords: bulk-close-intermittents, intermittent-failure)

Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → INCOMPLETE
Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=170303152&repo=mozilla-inbound&lineNumber=514 ++ ./src/mach artifact toolchain -v --authentication-file=c:/builds/relengapi.tok --tooltool-url https://tooltool.mozilla-releng.net/ --tooltool-manifest ./src/browser/config/tooltool-manifests/win32/releng.manifest --cache-dir 'C:\builds\tooltool_cache' public/build/clang.tar.bz2@XB4hwC6qRHCgOttw10md3Q public/build/rustc.tar.bz2@ffg6fNsQTt-itLqlby_50A 0:00.99 materialized mozbuild.action.tooltool.FileRecord(filename='mozmake.exe', size=266240, digest='bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869', algorithm='sha512', visibility=None) 0:00.99 materialized mozbuild.action.tooltool.FileRecord(filename='vs2017_15.6.0.zip', size=309779153, digest='f99285fa6328da3c21839adabe6fc4dd1a792bcb6048a491ba7617a8ce3d0e21f8d18e9de03b65c90ce5bf37073637ba7c1497fb21ce920b6794d2c9819b4f9e', algorithm='sha512', visibility=None) 0:00.99 materialized mozbuild.action.tooltool.FileRecord(filename='makecab.tar.bz2', size=297118, digest='196ac6a567c85559957dfe511c3d8654d23c94d5603259e19ccafe9d71e0e4ccee63ccc9a778f2699654b786cda54266108b7d4db543d01bb0b42545b4e6ec75', algorithm='sha512', visibility=u'public') 0:00.99 loaded manifest from file './src/browser/config/tooltool-manifests/win32/releng.manifest' Error running mach: ['artifact', 'toolchain', '-v', '--authentication-file=c:/builds/relengapi.tok', '--tooltool-url', 'https://tooltool.mozilla-releng.net/', '--tooltool-manifest', './src/browser/config/tooltool-manifests/win32/releng.manifest', '--cache-dir', 'C:\\builds\\tooltool_cache', 'public/build/clang.tar.bz2@XB4hwC6qRHCgOttw10md3Q', 'public/build/rustc.tar.bz2@ffg6fNsQTt-itLqlby_50A'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: ConnectionError: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host')) File "Z:\task_1522076890\src\python/mozbuild/mozbuild/mach_commands.py", line 1426, in artifact_toolchain record = ArtifactRecord(task_id, name) File "Z:\task_1522076890\src\python/mozbuild/mozbuild/mach_commands.py", line 1321, in __init__ get_artifact_url(task_id, 'public/chainOfTrust.json.asc')) File "Z:\task_1522076890\src\third_party/python/requests\requests\sessions.py", line 480, in get return self.request('GET', url, **kwargs) File "Z:\task_1522076890\src\third_party/python/requests\requests\sessions.py", line 468, in request resp = self.send(prep, **send_kwargs) File "Z:\task_1522076890\src\third_party/python/requests\requests\sessions.py", line 597, in send history = [resp for resp in gen] if allow_redirects else [] File "Z:\task_1522076890\src\third_party/python/requests\requests\sessions.py", line 195, in resolve_redirects **adapter_kwargs File "Z:\task_1522076890\src\third_party/python/requests\requests\sessions.py", line 576, in send r = adapter.send(request, **kwargs) File "Z:\task_1522076890\src\third_party/python/requests\requests\adapters.py", line 426, in send raise ConnectionError(err, request=request)
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 8 years ago7 years ago
Resolution: --- → INCOMPLETE
Status: REOPENED → RESOLVED
Closed: 7 years ago6 years ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → INCOMPLETE

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=246652615&repo=autoland&lineNumber=783

18:41:24 INFO - Error running mach:
18:41:24 INFO - ['artifact', 'toolchain', '-v', '--retry', '4', '--artifact-manifest', 'z:\task_1557939839\build\src\toolchains.json', '--tooltool-manifest', 'z:\task_1557939839\build\src\browser/config/tooltool-manifests/win32/releng.manifest', '--tooltool-url', 'https://tooltool.mozilla-releng.net/', '--authentication-file', 'c:\builds\relengapi.tok', '--cache-dir', 'c:/builds/tooltool_cache', 'public/build/clang.tar.bz2@B-YcuzPmTXiZ2KS3jker4A', 'public/build/rustc.tar.xz@f0mWnenOSoKz1aJ03xp7TQ', 'public/build/rust-size.tar.bz2@Cgp9FKpTQWihYcXupCwxkQ', 'public/build/cbindgen.tar.bz2@d68o7P_nSPizoSyCPi66pQ', 'public/build/nasm.tar.bz2@RsrbsihBSney0eWGUylMdg', 'public/build/node.tar.bz2@fkzAwH8RSV-zX9CPMJId4Q']
18:41:24 INFO - The error occurred in code that was called by the mach command. This is either
18:41:24 INFO - a bug in the called code itself or in the way that mach is calling it.
18:41:24 INFO - You can invoke |./mach busted| to check if this issue is already on file. If it
18:41:24 INFO - isn't, please use |./mach busted file| to report it. If |./mach busted| is
18:41:24 INFO - misbehaving, you can also inspect the dependencies of bug 1543241.
18:41:24 INFO - If filing a bug, please include the full output of mach, including this error
18:41:24 INFO - message.
18:41:24 INFO - The details of the failure are as follows:
18:41:24 INFO - ConnectionError: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host'))
18:41:24 INFO - File "z:\task_1557939839\build\src\python/mozbuild/mozbuild/mach_commands.py", line 1571, in artifact_toolchain
18:41:24 INFO - record = ArtifactRecord(task_id, name)
18:41:24 INFO - File "z:\task_1557939839\build\src\python/mozbuild/mozbuild/mach_commands.py", line 1478, in init
18:41:24 INFO - get_artifact_url(task_id, 'public/chain-of-trust.json'))
18:41:24 INFO - File "z:\task_1557939839\build\src\third_party/python/requests\requests\sessions.py", line 480, in get
18:41:24 INFO - return self.request('GET', url, **kwargs)
18:41:24 INFO - File "z:\task_1557939839\build\src\third_party/python/requests\requests\sessions.py", line 468, in request
18:41:24 INFO - resp = self.send(prep, **send_kwargs)
18:41:24 INFO - File "z:\task_1557939839\build\src\third_party/python/requests\requests\sessions.py", line 597, in send
18:41:24 INFO - history = [resp for resp in gen] if allow_redirects else []
18:41:24 INFO - File "z:\task_1557939839\build\src\third_party/python/requests\requests\sessions.py", line 195, in resolve_redirects
18:41:24 INFO - **adapter_kwargs
18:41:24 INFO - File "z:\task_1557939839\build\src\third_party/python/requests\requests\sessions.py", line 576, in send
18:41:24 INFO - r = adapter.send(request, **kwargs)
18:41:24 INFO - File "z:\task_1557939839\build\src\third_party/python/requests\requests\adapters.py", line 426, in send
18:41:24 INFO - raise ConnectionError(err, request=request)
18:41:24 ERROR - Return code: 1
18:41:24 ERROR - 1 not in success codes: [0]
18:41:24 WARNING - setting return code to 2
18:41:24 FATAL - Halting on failure while running ['c:\mozilla-build\python\python.exe', '-u', 'z:\task_1557939839\build\src\mach', 'artifact', 'toolchain', '-v', '--retry', '4', '--artifact-manifest', 'z:\task_1557939839\build\src\toolchains.json', '--tooltool-manifest', 'z:\task_1557939839\build\src\browser/config/tooltool-manifests/win32/releng.manifest', '--tooltool-url', 'https://tooltool.mozilla-releng.net/', '--authentication-file', 'c:\builds\relengapi.tok', '--cache-dir', 'c:/builds/tooltool_cache', 'public/build/clang.tar.bz2@B-YcuzPmTXiZ2KS3jker4A', 'public/build/rustc.tar.xz@f0mWnenOSoKz1aJ03xp7TQ', 'public/build/rust-size.tar.bz2@Cgp9FKpTQWihYcXupCwxkQ', 'public/build/cbindgen.tar.bz2@d68o7P_nSPizoSyCPi66pQ', 'public/build/nasm.tar.bz2@RsrbsihBSney0eWGUylMdg', 'public/build/node.tar.bz2@fkzAwH8RSV-zX9CPMJId4Q']
18:41:24 FATAL - Running post_fatal callback...
18:41:24 FATAL - Exiting 2
18:41:24 INFO - [mozharness: 2019-05-15 18:41:24.051000Z] Finished build step (failed)
18:41:24 INFO - Running post-run listener: _parse_build_tests_ccov
18:41:24 INFO - Running post-run listener: _shutdown_sccache
18:41:24 INFO - Running post-run listener: _summarize
18:41:24 ERROR - # TBPL FAILURE #
18:41:24 INFO - [mozharness: 2019-05-15 18:41:24.052000Z] FxDesktopBuild summary:
18:41:24 ERROR - # TBPL FAILURE #
[taskcluster 2019-05-15T18:41:24.068Z] Exit Code: 2
[taskcluster 2019-05-15T18:41:24.068Z] User Time: 15.625ms
[taskcluster 2019-05-15T18:41:24.068Z] Kernel Time: 0s
[taskcluster 2019-05-15T18:41:24.068Z] Wall Time: 30.5093656s
[taskcluster 2019-05-15T18:41:24.068Z] Result: FAILED
[taskcluster 2019-05-15T18:41:24.068Z] === Task Finished ===
[taskcluster 2019-05-15T18:41:24.068Z] Task Duration: 8m14.4611491s
[taskcluster 2019-05-15T18:41:24.635Z] Uploading artifact public/logs/localconfig.json from file logs\localconfig.json with content encoding "gzip", mime type "application/octet-stream" and expiry 2020-05-14T18:32:35.757Z
[taskcluster 2019-05-15T18:41:25.250Z] Uploading artifact public/logs/certified.log from file generic-worker\certified.log with content encoding "gzip", mime type "text/plain; charset=utf-8" and expiry 2020-05-14T18:32:35.757Z
[taskcluster 2019-05-15T18:41:25.690Z] Uploading artifact public/chain-of-trust.json from file generic-worker\chain-of-trust.json with content encoding "gzip", mime type "text/plain; charset=utf-8" and expiry 2020-05-14T18:32:35.757Z
[taskcluster 2019-05-15T18:41:26.068Z] Uploading artifact public/chain-of-trust.json.sig from file generic-worker\chain-of-trust.json.sig with content encoding "gzip", mime type "application/octet-stream" and expiry 2020-05-14T18:32:35.757Z
[taskcluster 2019-05-15T18:41:26.655Z] Uploading redirect artifact public/logs/live.log to URL https://queue.taskcluster.net/v1/task/XXZ5V0qHSTa4C9l3QPs20A/runs/0/artifacts/public/logs/live_backing.log with mime type "text/plain; charset=utf-8" and expiry 2020-05-14T18:32:35.757Z
[taskcluster:error] exit status 2

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
See Also: → 1773550
Assignee: nobody → gbrown
Status: REOPENED → RESOLVED
Closed: 6 years ago3 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.