Closed Bug 1233611 Opened 8 years ago Closed 8 years ago

AWS instance fails to clone project repo 'cypress'

Categories

(Developer Services :: General, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: hwine, Assigned: hwine)

References

Details

(Whiteboard: [vcs-sync])

Attachments

(1 file)

The cypress repo can no longer be cloned on the AWS instance processing vcs-sync for gecko-projects in developer services.

The cypress repo was just reset in bug 1222301 and much additional code added to it since. It is now the largest hg repo being converted.

Traceback reports for the failed clone are all similar to:
    00:12:41     INFO -    File "/opt/vcs2vcs/vcssync2/build/venv/lib64/python2.7/site-packages/mercurial/httppeer.py", line 23, in zgenerator
    00:12:41     INFO -      raise IOError(None, _('connection ended unexpectedly'))
    00:12:41     INFO -  IOError: [Errno None] connection ended unexpectedly
    00:12:41    ERROR -  abort: connection ended unexpectedly
    00:12:41    ERROR -  Automation Error: hg not responding
    00:12:41    ERROR - Return code: 255
After some research[1], this appears to be due to a combination of:
  a) older mercurial client version used in vcs-sync (2.6.3) (per :gps)
  b) size of repository-to-be-cloned (cypress is merge of
     mozilla-central and comm-central)
  c) size of ec2 instance used by dev-services instance
     (releng used larger and did not have this problem)

This combination leads to the decompressing of the bundle taking so long that the TCP connection times out.

The quickest way to address the issue is to add the "--uncompressed" option to the "hg clone" command for the repository. Manual testing shows this works.


[1] https://hglabhq.com/blog/2014/6/20/working-with-mercurial-over-unreliable-connections was most helpful
Add an optional config parameter to specify additional clone options.

Tested by running in production for gecko-projects :/ but works without issue for both projects with new config value and projects without config value.

Not (yet) tested for the other conversions. The impact will be easy to test after this is landed on default, before merge to production branch.
Assignee: nobody → hwine
Status: NEW → ASSIGNED
Comment on attachment 8699805 [details] [diff] [review]
add clone options

On production branch after testing no impact on l10n conversions.
  https://hg.mozilla.org/build/mozharness/rev/9901bc9e8ee4
Attachment #8699805 - Flags: review+
Attachment #8699805 - Flags: checked-in+
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
See Also: → 1253068
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: