Open Bug 1362845 Opened 7 years ago Updated 7 years ago

Pushing to mozilla-central fails with: remote: Write failed: Connection reset by peer

Categories

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Product:
Developer Services
Component:
Mercurial: hg.mozilla.org
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

People

(Reporter: aryx, Unassigned)

Details

Today, pushing the merges from autoland or mozilla-inbound to mozilla-central failed.

Output for autoland merge (26a2a52b1108 from autoland on top of centrals's tip 37a5b7f6f101):

$ hg push
pushing to ssh://hg.mozilla.org/mozilla-central/
searching for changes
remote: Write failed: Connection reset by peer
abort: Invalid argument

Output with --debug:

$ hg push --debug
automatically setting Bugzilla API Key auth https://reviewboard-hg.mozilla.org
pushing to ssh://hg.mozilla.org/mozilla-central/
running ssh hg.mozilla.org "hg -R mozilla-central/ serve --stdio"
sending hello command
sending between command
remote: 376
remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps unbundle=HG10GZ,HG10BZ,HG10UN clonebundles pushlog
remote: 1
query 1; heads
sending batch command
searching for changes
all remote heads known locally
preparing listkeys for "phases"
sending listkeys command
received listkey for "phases": 15 bytes
checking for updated bookmarks
preparing listkeys for "bookmarks"
sending listkeys command
received listkey for "bookmarks": 0 bytes
sending branchmap command
sending branchmap command
preparing listkeys for "bookmarks"
sending listkeys command
received listkey for "bookmarks": 0 bytes
75 changesets found
list of changesets:
5c219b4312a787fc2f0384959bd94c7037e8c054
cd70682a78180ac48568a9d2b8dcd345627d40d3
af0de3170a13ef58dfe923b86e2c1b998793f5ab
400dfd6e6f04e2863c470c45e9689cafe9cd49e5
fa46b44d5629b09f4e0788bbaed53d924eed02c4
ae723b6d1271aa41bccaabea9f5c6e8b3c353143
c926817dea602dd5ec1e836607c13effc24d90b4
9201d345a1d5cef6b5ffb91ed27983403cc43865
3c1b426a5cce74b34841fc608842518c8746f8f3
95211a49619145612d3e55e95b9f596bd649f0cb
92ec85d184950fe93974716b8cac3c9273e4e3aa
952c5438a347b382be5f6ac516365c9a299818bd
70db5f9c852d04d2d73e879f80385c2c484c2f17
3398e932771022bd827367d5409d1457d66381c1
ea693af1b5fbb095654edd0a922fee7ce2f4b536
e9cfbd2b4fe2654522d8d325228a529571f4a55f
69beada73635a06d1577c73876e708b067a3ab35
c57f94d10914aa00968ac3d61f85fbf224b639c0
95e800f66f2b5b8f99cc7c391fbc4d177a693e6f
d5f91c51a3ab24ccf89e3a06d77007a679ae016a
5c767e2e3dedaded01efd9eb4f91aeda51717b1b
2e4c64214e31c3a759918133fbfb6db25f68e33f
53e95295210a0364bdf9a8f076b61ef4cb2059f5
b363981633ad2707328a7828e17abcaf37af00a5
355a23c6c2bd2806475c17e36267a189a87901f9
bb299d98bdc428ffe9ae3aef5ace8cb03e27052b
de6c23251b7585559dda4c12eca980bb58be3e15
62b03dd84117bf8795643dfe96aa0a50119e6c2c
832a7bbcea613eff31d8afa8930034af48e34793
b820c22816054fb93270d66375862913054bee7c
d330e31dcb2345bdf49352249fe533721215cd15
ed74f44b40c1cc58bfe3cd628fc5834336be6081
3e44cbaec84b4412136e9b36a8403a88cc6b7742
3633cb2d5260126d77c82fb61bc9e8548032ee38
20b0f18b27452611e3204eecd29efa7166404104
24b55221df23c82bd45cccbb0b3c7596bd9e5323
d48d678f5345fa053dfc9aadaddfbe610cec379f
577a703947bfac765f5ec090b4d8cfac75105051
2ce843dc8642f29e6e68b36da80b0c8194e07693
e6ff9a1af4b3642b3f0caadfda28f23c1ddb08f9
90c2b464f60c75d7c5403da527ef9c12b6530ddd
1c93dbb91910f1c7b6f07e794ac2af7a215e4522
5a4de575944b503ec8d3f5e301200c3abc69e020
a717a61128accd351be6554bb747ef34f779c59b
a88c73f843dc199a6f584cdc2829d05369de1ce9
cbd3b49c0c2912567d66421bce8d106611aea598
4cc80402cf1cd907de1d7d124e620f07171d402f
64bbd52dc60dcb6eb384f934182dfb9124344dca
9a8a5130ea32a0c9423e65f4434b751aa6175c56
69a52aed9d22f90a417b1d6596040a7a2c2be8cb
2eff8fd9e6084155d2d75adba903874bb6d576ea
c88a3c2a1d7cd0d57c23da50d3f0b17d51a87497
443b66118588b3d6ba370723c948e88c038515c3
34bacb8599bd02f3fe1d875430e1f19e7c261b28
d5880ff564eb357a3c855c836d6c69911f6d6422
1b4c181052c75d74384d360da9a76a94822a5a5a
05dc4f814accd4f36eaeb30df5e77f6209edac13
c8deb87d938b95382cbc00a7261f74c1e6a6308a
b532001b111626c145ada4ce9472eee67c39d793
323cb7b05a81eb19407cf2a3b8241cc68c96c36c
80d2096d675684e6d06787660069b35f1b6ac7fc
d5cc139abcfdf2b785fe68ee54afdd9283ccfbc2
e3bceb6eb287275e123cb92c487696c6c7325096
bce76b2df8cc5a22cadfc3c06c61b94533308586
3d902e0432619099e4d292b1395e6f3af182525f
2056ed360550bae11cd62974613ac49ab8cc64bf
ebd5f2bb5c5948ae2348e62d3b71c018b0669934
d34041d4dfe96a6872d2c4a569e0c428fa91a371
1fb2d6e0aa2d82c2db246ecd75f7225fecc449ed
7a2fb60f21ad7d2f6b06bad7f6e8b87c459dad7c
d7331d0d42525673d2f6f503494253bbe03bd1d6
43aad7ae71a62c63b1800212de603b26067d8ae0
b265fdf79862eb2f35c498a744d9f480714a88ed
d780c3ea652df28ea24c647f8145c10f649133c3
26a2a52b11089ba3d0c7845cfa5ff99bbde62962
sending unbundle command
bundle2-output-bundle: "HG20", 4 parts total
bundle2-output-part: "replycaps" 155 bytes payload
bundle2-output-part: "check:heads" streamed payload
bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
remote: Write failed: Connection reset by peer
abort: Invalid argument
bundle2-generatorexit

The push of an inbound merge is slightly different:
new line |bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload| and changed line |abort: stream ended unexpectedly (got 0 bytes, expected 4)|

A restart of the machine didn't solve the issue.

Last successful push to central was yesterday: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=0b255199db9d6a6f189b89b7906f99155bde3726&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable
Last successful push to autoland was today: https://hg.mozilla.org/integration/autoland/rev/74ff52c38236a889ac008c612e4754f5dc68602a
Gregory, can you take a look at this, please? The automated HPKP and HSTS updates on mozilla-central still work, so is this an issue at my end? Thank you.
Component: Buildduty → Mercurial: hg.mozilla.org
Flags: needinfo?(gps)
Product: Release Engineering → Developer Services
QA Contact: bugspam.Callek
Iris was able to push earlier today. -> Severity: normal
Severity: blocker → normal
Something was likely idle for too long causing connection drop. Unsure exactly what.

We could potentially increase idle timeout on the SSH server if people are seeing this frequently.
Flags: needinfo?(gps)
You need to log in before you can comment on or make changes to this bug.