1499204 - Set ClientAliveInterval on SSH server

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect, P1)

Description

[Gregory Szorc [:gps]]

When aki attempted to push mozilla-beta to hg.mozilla.org today, SSH connections kept getting dropped while waiting for hooks to complete.

I think I successfully fingered the dropping of idle SSH connections as the culprit. (SSH will drop connections if no data has been exchanged for a while.)

On clients, you can enable ServerAliveInterval to send data every N seconds to keep the connection alive. Servers can enable ClientAliveInterval.

I hacked up the sshd_config for the hg server to add the following:

ClientAliveInterval 10
ClientAliveCountMax 6

aki's next push didn't drop.

We should make this change permanent so the server ensures idle SSH connections don't drop.

The settings above will drop the connection if the client doesn't respond to the pings after 60s. This seems reasonable. But I kinda pulled that number out of thin air. Not sure if others have a strong opinion on a better value...

Comment 1

[Gregory Szorc [:gps]]

Attachment: ansible/hg-ssh-server: enable ClientAliveInterval (bug 1499204); r?sheehan

This will prevent clients from timing out due to no output from
server.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/c5cabbbc31a5
ansible/hg-ssh-server: enable ClientAliveInterval ; r=sheehan