Closed
Bug 1317594
Opened 8 years ago
Closed 7 years ago
Intermittent abort: error applying bundle
Categories
(Developer Services :: Mercurial: robustcheckout, defect)
Developer Services
Mercurial: robustcheckout
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: intermittent-bug-filer, Assigned: gps)
References
(Blocks 1 open bug)
Details
(Keywords: intermittent-failure)
Attachments
(4 files)
Filed by: philringnalda [at] gmail.com https://treeherder.mozilla.org/logviewer.html#?job_id=6633280&repo=autoland https://queue.taskcluster.net/v1/task/fovCB_XTRsOxhZPBOzRWvg/runs/0/artifacts/public%2Flogs%2Flive_backing.log
|
||
Comment 1•8 years ago
|
||
Greg, looks like this is an issue with the robust checkout?
Flags: needinfo?(gps)
|
Assignee | |
Comment 2•8 years ago
|
||
The log says a connection to https://s3-external-1.amazonaws.com is failing. The only thing you can blame robustcheckout for is not retrying in case of this error. I don't think I've seen HTTP requests to S3 fail like this before. First time for everything.
Flags: needinfo?(gps)
|
|
||
Comment 3•8 years ago
|
||
Oh, indeed, we had some similar issues with mozharness.zip failing around the same time (bug 1266624). Probably a common cause.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
| Comment hidden (Intermittent Failures Robot) |
|
||
Comment 5•8 years ago
|
||
Still a thing, but with lower frequency: https://public-artifacts.taskcluster.net/SZRfPJKlSau5A5s1ovxwAQ/0/public/logs/live_backing.log
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
|
|
||
Comment 6•8 years ago
|
||
Looks like it's time for some retry in robustcheckout
Component: General → Mercurial: robustcheckout
Product: Taskcluster → Developer Services
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
Assignee | |
Comment 16•7 years ago
|
||
Example log output: [vcs 2017-04-03T18:17:33.613036Z] executing ['hg', 'robustcheckout', '--sharebase', '/home/worker/checkouts/hg-store', '--purge', '--upstream', 'https://hg.mozilla.org/mozilla-unified', '--revision', 'caee52f2863b8c46da0a28d593d9efff2c09465a', 'https://hg.mozilla.org/integration/autoland/', '/home/worker/workspace/build/src'] [vcs 2017-04-03T18:17:33.676151Z] ensuring https://hg.mozilla.org/integration/autoland/@caee52f2863b8c46da0a28d593d9efff2c09465a is available at /home/worker/workspace/build/src [vcs 2017-04-03T18:17:33.676279Z] (cloning from upstream repo https://hg.mozilla.org/mozilla-unified) [vcs 2017-04-03T18:17:34.381987Z] (sharing from new pooled repository 8ba995b74e18334ab3707f27e9eb8f4e37ba3d29) [vcs 2017-04-03T18:19:42.179885Z] applying clone bundle from https://s3-external-1.amazonaws.com/moz-hg-bundles-us-east-1/mozilla-unified/66146205625406fd7eabd6b03c8b4c7ec464c3a1.packed1-gd.hg [vcs 2017-04-03T18:21:49.415396Z] error fetching bundle: Connection timed out [vcs 2017-04-03T18:21:49.417022Z] abort: error applying bundle [vcs 2017-04-03T18:21:49.417073Z] (if this error persists, consider contacting the server operator or disable clone bundles via "--config ui.clonebundles=false") I think this is a matter of adding that "Connection timed out" message to our list of errors to retry after encountering.
|
|
Assignee | |
Comment 17•7 years ago
|
||
I'll code up some patches.
Assignee: nobody → gps
Status: REOPENED → ASSIGNED
| Comment hidden (mozreview-request) |
| Comment hidden (mozreview-request) |
| Comment hidden (mozreview-request) |
| Comment hidden (mozreview-request) |
|
||
Comment 22•7 years ago
|
||
| mozreview-review | ||
Comment on attachment 8856701 [details] robustcheckout: add test demonstrating connection failure (bug 1317594); https://reviewboard.mozilla.org/r/128636/#review131264
Attachment #8856701 -
Flags: review?(glob) → review+
|
|
||
Comment 23•7 years ago
|
||
| mozreview-review | ||
Comment on attachment 8856702 [details] robustcheckout: factor network failure handling into own function (bug 1317594); https://reviewboard.mozilla.org/r/128638/#review131280
Attachment #8856702 -
Flags: review?(glob) → review+
|
|
||
Comment 24•7 years ago
|
||
| mozreview-review | ||
Comment on attachment 8856703 [details] robustcheckout: refactor handlepullabort to handle multiple types (bug 1317594); https://reviewboard.mozilla.org/r/128640/#review131422
Attachment #8856703 -
Flags: review?(glob) → review+
|
|
||
Comment 25•7 years ago
|
||
| mozreview-review | ||
Comment on attachment 8856704 [details] robustcheckout: retry after socket errors (bug 1317594); https://reviewboard.mozilla.org/r/128642/#review131424
Attachment #8856704 -
Flags: review?(glob) → review+
|
||
Comment 26•7 years ago
|
||
Pushed by gszorc@mozilla.com: https://hg.mozilla.org/hgcustom/version-control-tools/rev/8318beaacec6 robustcheckout: add test demonstrating connection failure ; r=glob https://hg.mozilla.org/hgcustom/version-control-tools/rev/6f25918d6b2b robustcheckout: factor network failure handling into own function ; r=glob https://hg.mozilla.org/hgcustom/version-control-tools/rev/03efe6e45246 robustcheckout: refactor handlepullabort to handle multiple types ; r=glob https://hg.mozilla.org/hgcustom/version-control-tools/rev/e0d30b04dac6 robustcheckout: retry after socket errors ; r=glob
Status: ASSIGNED → RESOLVED
Closed: 8 years ago → 7 years ago
Resolution: --- → FIXED
|
|
||
Comment 27•7 years ago
|
||
Pushed by gszorc@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/987931d9e607 Vendor latest version of robustcheckout extension; r=me
|
|
||
Comment 28•7 years ago
|
||
Pushed by gszorc@mozilla.com: https://hg.mozilla.org/hgcustom/version-control-tools/rev/249a47720ddc robustcheckout: restore compatibility with Mercurial 3.7
|
|
||
Comment 29•7 years ago
|
||
Pushed by gszorc@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/f124872fb221 Vendor latest robustcheckout extension; r=me
|
||
Comment 30•7 years ago
|
||
| bugherder uplift | ||
https://hg.mozilla.org/releases/mozilla-aurora/rev/1c79bcfea679
|
|
||
Comment 31•7 years ago
|
||
| bugherder uplift | ||
https://hg.mozilla.org/releases/mozilla-beta/rev/cfdbc0969cac
|
|
||
Comment 32•7 years ago
|
||
| bugherder uplift | ||
https://hg.mozilla.org/releases/mozilla-esr52/rev/15f63401caef
|
|
||
Comment 33•7 years ago
|
||
| bugherder uplift | ||
https://hg.mozilla.org/releases/mozilla-release/rev/cfdbc0969cac
|
||
Comment 34•7 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/987931d9e607 https://hg.mozilla.org/mozilla-central/rev/f124872fb221
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
You need to log in
before you can comment on or make changes to this bug.
Description
•