Closed Bug 1451432 Opened 7 years ago Closed 7 years ago

Intermittent Could not install python package: /builds/worker/workspace/build/venv/bin/pip install --timeout 120 --no-index --find-links http://pypi.pub.build.mozilla.org/pub --trusted-host pypi.pub.build.mozilla.org jsonschema==2.5.1 failed after 5 tries

Categories

(Release Engineering :: General, defect, P5)

Product:
Release Engineering
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: intermittent-failure)

This may be related to bug 1399926 - we may have bounced the web daemon, creating a short hiccup in service. If that's the case, we probably won't see this again.
See Also: → 1444728, 1444663, 1452923
In addition to the likely dups (see also's), I see several of these reported in bug 1411358 each day: persistent pip install failures eventually cause task timeouts.
Blocks: 1411358
From bug 1411358: https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=172940658&lineNumber=576-583 [task 2018-04-10T19:39:12.524Z] 19:39:12 INFO - Collecting psutil>=3.1.1 [task 2018-04-10T19:41:12.642Z] 19:41:12 INFO - Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f36cb3eb490>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T19:43:13.419Z] 19:43:13 INFO - Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f36cb3eb610>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T19:45:14.644Z] 19:45:14 INFO - Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f36cb3eb790>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T19:47:16.754Z] 19:47:16 INFO - Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f36cb3eb910>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T19:49:21.023Z] 19:49:21 INFO - Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f36cb3eba90>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T19:51:21.190Z] 19:51:21 INFO - Could not find a version that satisfies the requirement psutil>=3.1.1 (from versions: ) [task 2018-04-10T19:51:21.191Z] 19:51:21 INFO - No matching distribution found for psutil>=3.1.1 ... [task 2018-04-10T19:51:21.780Z] 19:51:21 INFO - Collecting mozsystemmonitor==0.3 [task 2018-04-10T19:53:21.872Z] 19:53:21 INFO - Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9fcc3f4d0>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T19:55:22.527Z] 19:55:22 INFO - Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9fcc3f650>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T19:57:23.788Z] 19:57:23 INFO - Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9fcc3f7d0>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T19:59:25.873Z] 19:59:25 INFO - Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9fcc3f950>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T20:01:29.971Z] 20:01:29 INFO - Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9fcc3fad0>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T20:03:30.142Z] 20:03:30 INFO - Could not find a version that satisfies the requirement mozsystemmonitor==0.3 (from versions: ) [task 2018-04-10T20:03:30.143Z] 20:03:30 INFO - No matching distribution found for mozsystemmonitor==0.3 [task 2018-04-10T20:03:30.159Z] 20:03:30 WARNING - Return code: 1 ... [task 2018-04-10T20:46:58.540Z] 20:46:58 INFO - Collecting jsonschema==2.5.1 [task 2018-04-10T20:48:58.862Z] 20:48:58 INFO - Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f33ce3ad490>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T20:50:59.455Z] 20:50:59 INFO - Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f33ce3ad610>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T20:53:00.676Z] 20:53:00 INFO - Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f33ce3ad790>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T20:55:02.996Z] 20:55:02 INFO - Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f33ce3ad910>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T20:57:07.186Z] 20:57:07 INFO - Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f33ce3ada90>, 'Connection to pypi.pub.build.mozilla.org timed out. (connect timeout=120.0)')': /pub [task 2018-04-10T20:59:07.642Z] 20:59:07 INFO - Could not find a version that satisfies the requirement jsonschema==2.5.1 (from versions: ) [task 2018-04-10T20:59:07.643Z] 20:59:07 INFO - No matching distribution found for jsonschema==2.5.1 [task 2018-04-10T20:59:07.659Z] 20:59:07 WARNING - Return code: 1 [task 2018-04-10T20:59:07.660Z] 20:59:07 INFO - retry: Failed, sleeping 300 seconds before retrying ... [taskcluster:error] Task timeout after 5400 seconds. Force killing container. [taskcluster 2018-04-10 21:08:00.439Z] === Task Finished === [taskcluster 2018-04-10 21:08:00.440Z] Unsuccessful task run with exit code: -1 completed in 5729.556 seconds
:catlee - Can you find someone to look at this?
Flags: needinfo?(catlee)
:sidler, is port 80 still active on our internal pypi?
Flags: needinfo?(catlee) → needinfo?(sidler)
See Also: → 1451193
Yes port 80 is still active on our internal pypi; verified via curl.
Flags: needinfo?(sidler)
I don't see this happening any longer.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.