Broken pipe when pushing to try via ./mach try or ./mach try chooser on Mac
Categories
(Firefox Build System :: Try, defect)
Tracking
(Not tracked)
People
(Reporter: emmamalysz, Assigned: mhentges)
References
(Blocks 1 open bug)
Details
Attachments
(2 files)
I've recently been unable to push to try. I've tried running ./mach bootstrap
and ./mach vcs-setup
, but that doesn't seem to fix the issue.
Here's my full output
searching for changes
remote: packet_write_wait: Connection to 63.245.208.203 port 22: Broken pipe
temporary commit removed, repository restored
abort: unexpected response: empty string
Error running mach:
['try']
The details of the failure are as follows:
subprocess.CalledProcessError: Command '('/usr/local/bin/hg', 'push-to-try', '-m', 'Tasks automatically selected.\n\nPushed via `mach try auto`')' returned non-zero exit status 255.
File "/Users/emmamalysz/Desktop/firefox-source/tools/tryselect/mach_commands.py", line 227, in try_default
return self._mach_context.commands.dispatch(
File "/Users/emmamalysz/Desktop/firefox-source/python/mach/mach/registrar.py", line 161, in dispatch
return self._run_command_handler(handler, context, **kwargs)
File "/Users/emmamalysz/Desktop/firefox-source/python/mach/mach/registrar.py", line 107, in _run_command_handler
result = fn(**kwargs)
File "/Users/emmamalysz/Desktop/firefox-source/tools/tryselect/mach_commands.py", line 365, in try_auto
return self.run(**kwargs)
File "/Users/emmamalysz/Desktop/firefox-source/tools/tryselect/mach_commands.py", line 197, in run
return mod.run(**kwargs)
File "/Users/emmamalysz/Desktop/firefox-source/tools/tryselect/selectors/auto.py", line 107, in run
return push_to_try(
File "/Users/emmamalysz/Desktop/firefox-source/tools/tryselect/push.py", line 213, in push_to_try
vcs.push_to_try(commit_message)
File "/Users/emmamalysz/Desktop/firefox-source/python/mozversioncontrol/mozversioncontrol/__init__.py", line 482, in push_to_try
subprocess.check_call(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
I tried disabling clang-format
and js-format
, but that didn't seem to fix the issue.
Any help would be much appreciated!
Assignee | ||
Comment 1•2 years ago
•
|
||
hg push-try-try
is printing out the following:
remote: packet_write_wait: Connection to 63.245.208.203 port 22: Broken pipe
temporary commit removed, repository restored
** Unknown exception encountered with possibly-broken third-party extension clang-format
** which supports versions 5.3 of Mercurial.
** Please disable clang-format and try your action again.
** If that fixes the bug please report it to https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox%20Build%20System&component=Lint%20and%20Formatting
** Python 3.8.5 (default, Jul 21 2020, 10:42:08) [Clang 11.0.0 (clang-1100.0.33.17)]
** Mercurial Distributed SCM (version 5.5.2)
** Extensions loaded: absorb, histedit, purge, rebase, evolve, fsmonitor, blackbox, firefoxtree, push-to-try, clang-format, js-format
Assignee | ||
Updated•2 years ago
|
Assignee | ||
Comment 2•2 years ago
|
||
Rebooting to Mac, let me see if I can reproduce this.
If you go to central
with a clean checkout and you ./mach try ...
, can you still reproduce the problem?
Assignee | ||
Updated•2 years ago
|
Reporter | ||
Comment 3•2 years ago
|
||
```
Reporter | ||
Comment 4•2 years ago
|
||
```
Reporter | ||
Comment 5•2 years ago
|
||
Thanks for the help :mhentges!
At least for now, the problem seems to have gone away. Mitchell suggested updating mercurial to version 5.6 and pulling a fresh copy of central, and I cannot currently reproduce this issue. I also removed all of my untracked files
Assignee | ||
Comment 6•2 years ago
|
||
I'm not able to reproduce this, but I'm glad that it's gone away for some reason or another :)
One theory is that this is could've been an intermittent timeout issue. If so, if this happens again, adding the following to ~/.ssh/config
may be a helpful workaround:
Host hg.mozilla.org
ServerAliveInterval 30
ServerAliveCountMax 5
Also, if this issue is encountered in the future by some internet denizen, and the above SSH config chunk fixes the issue, please mention so in this bug :)
Assignee | ||
Comment 7•2 years ago
|
||
Note: we did some more investigation and tried with the following options:
Host hg.mozilla.org
ServerAliveInterval 120
ServerAliveCountMax 5
IPQoS=throughput
TCPKeepAlive no
but, this still didn't work.
Also worth noting:
- This could be reproduced with a raw
hg push ssh://hg.mozilla.org/try -r <try revision>
. - This could be reproduced by SSH-ing into any server and pushing data quickly (e.g.:
cat /dev/urandom
, SFTP-ing a large file). - If Emma used cell data (tethering) instead of wifi, the issue went away.
- In the last week, an ISP technician came by to update the local router. This could've changed a network config, thereby causing this failure.
The solution in this case is that the ISP will be notified that "when SSHing from the local network, pushing data quickly (e.g.: sending files) causes a 'broken pipe' error".
Description
•