Closed Bug 463113 Opened 13 years ago Closed 12 years ago

ssh batch mode

Categories

(Release Engineering :: General, defect, P4)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: catlee, Assigned: catlee)

References

Details

(Whiteboard: [simple][oldbug])

Attachments

(3 files, 1 obsolete file)

ssh / scp should be run with batch mode on the buildbot slaves.  Batch mode prevents ssh asking for a password if no public key is accepted.  Right now if the keys aren't set up properly we wait 20 minutes before killing off the process.
what is to be run in batch mode?
sh -C "scp other arguments"?
scp has a '-B' option, or you can use the generic -o 'BatchMode=yes' format for scp or ssh.

This just prevents ssh / scp from asking for a password.
with this as I understand it does make the step fail if the keys are not set properly in that slave rather than timing out. is this correct?
(In reply to comment #3)
> with this as I understand it does make the step fail if the keys are not set
> properly in that slave rather than timing out. is this correct?

yes, that's right.
Component: Release Engineering → Release Engineering: Future
Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.
Component: Release Engineering: Future → Release Engineering
Priority: -- → P3
Whiteboard: [simple]
Priority: P3 → P5
Fixable via a ~/.ssh/config setting?

BatchMode "yes"

or

PasswordAuthentication "no"
Assignee: nobody → catlee
We'll need to audit known_hosts at some point.  I copied from staging slaves, and deleted a bunch of miscellaneous hosts from known_hosts.

I added 'BatchMode yes' to the top of all .ssh/config files.
Attachment #468424 - Flags: review?(nrthomas)
Attachment #468424 - Flags: review?(nrthomas) → review?(bhearsum)
Comment on attachment 468424 [details] [diff] [review]
Sync up config/known_hosts on other platforms

Looks fine to me. Sorry for misleading you on the mac-should-work-in-cltbld.pp thing.
Attachment #468424 - Flags: review?(bhearsum) → review+
Priority: P5 → P4
Whiteboard: [simple] → [simple][oldbug]
Comment on attachment 468424 [details] [diff] [review]
Sync up config/known_hosts on other platforms

changeset:   217:b581524de8a6
Attachment #468424 - Flags: checked-in+
Attachment #471974 - Flags: review?(nrthomas)
Depends on: 593707
Comment on attachment 471974 [details] [diff] [review]
OPSI packages for ssh config/known hosts

>diff --git a/ssh-config-staging/CLIENT_DATA/config b/ssh-config-staging/CLIENT_DATA/config
>+IdentityFile /home/cltbld/.ssh/id_dsa

cltbld@WIN32-SLAVE20 ~
$ ls /home/cltbld/.ssh/
ls: /home/cltbld/.ssh/: No such file or directory

Need to translate the paths for win32. Looks like we have this sort of thing at the moment:
IdentityFile "/c/Documents and Settings/cltbld/.ssh/id_dsa"

Same for ssh-config/CLIENT_DATA/config. Did you test making connections after deploying the staging package ?

>diff --git a/ssh-config/CLIENT_DATA/config b/ssh-config/CLIENT_DATA/config
>+Host cvs.mozilla.org
>+IdentityFile /home/cltbld/.ssh/cvs
>+User stgbld

I've lost track of what stgbld is for, but this seems wrong production. Somehow it has no effect, eg 
 cvs -d :ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R12 -d build mozilla/tools/patcher
still works on the linux boxes this has been deployed to.

Otherwise looks fine, r+ if you fix the paths in config.
Attachment #471974 - Flags: review?(nrthomas) → review+
(In reply to comment #11)
> Comment on attachment 471974 [details] [diff] [review]
> OPSI packages for ssh config/known hosts
> 
> >diff --git a/ssh-config-staging/CLIENT_DATA/config b/ssh-config-staging/CLIENT_DATA/config
> >+IdentityFile /home/cltbld/.ssh/id_dsa
> 
> cltbld@WIN32-SLAVE20 ~
> $ ls /home/cltbld/.ssh/
> ls: /home/cltbld/.ssh/: No such file or directory
> 
> Need to translate the paths for win32. Looks like we have this sort of thing at
> the moment:
> IdentityFile "/c/Documents and Settings/cltbld/.ssh/id_dsa"
> 
> Same for ssh-config/CLIENT_DATA/config. Did you test making connections after
> deploying the staging package ?

mw32-ix-slave01 had /home/cltbld/.ssh referenced in its original ssh config...which looks like it wasn't working.

> 
> >diff --git a/ssh-config/CLIENT_DATA/config b/ssh-config/CLIENT_DATA/config
> >+Host cvs.mozilla.org
> >+IdentityFile /home/cltbld/.ssh/cvs
> >+User stgbld
> 
> I've lost track of what stgbld is for, but this seems wrong production. Somehow
> it has no effect, eg 
>  cvs -d :ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R12 -d
> build mozilla/tools/patcher
> still works on the linux boxes this has been deployed to.

I also don't know what it's for.  Should I remove it altogether?
if the main consumers of the upload and uploadsymbols targets are automated systems we could make the default action be to upload in batch mode regardless of client options.

(untested patch)
Attachment #472802 - Attachment is patch: true
Attachment #472802 - Attachment mime type: application/octet-stream → text/plain
(In reply to comment #12)
> I also don't know what it's for.  Should I remove it altogether?

Perhaps we should, after looking at all the cvs references in buildbotcustom. I bet we're using :ext:cltbld@cvs just about everywhere, but haven't checked how that interacts with the .ssh/config. A random spot check of production machines finds cvs.pub present, with cltbld@staging-build-console as the generating host. Goes against our policy to not mix production and staging keys.
(In reply to comment #14)
> (In reply to comment #12)
> > I also don't know what it's for.  Should I remove it altogether?
> 
> Perhaps we should, after looking at all the cvs references in buildbotcustom. I
> bet we're using :ext:cltbld@cvs just about everywhere, but haven't checked how
> that interacts with the .ssh/config. A random spot check of production machines
> finds cvs.pub present, with cltbld@staging-build-console as the generating
> host. Goes against our policy to not mix production and staging keys.

If we're using explicit username options to CVS, those override the user in .ssh/config.

I'll remove them from the opsi configs.
The two modifications we talked about (path to home dir, dropping staging keys) aren't implemented anywhere on staging-opsi. Removing blocking on downtime bug
No longer blocks: releng-downtime
Attachment #471974 - Attachment is obsolete: true
Attachment #476840 - Flags: review?(nrthomas)
Comment on attachment 476840 [details] [diff] [review]
OPSI packages for ssh config/known hosts

>diff --git a/ssh-config-staging/CLIENT_DATA/config b/ssh-config-staging/CLIENT_DATA/config
>@@ -0,0 +1,5 @@
>+BatchMode yes
>+IdentityFile "/c/Documents and Settings/cltbld/.ssh/id_dsa"

I think I'd prefer to use ~ instead of "/c/Documents and Settings/cltbld/", since it should allow us to use the same config on all platforms. I tested that it works on win32-slave04.
Attachment #476840 - Flags: review?(nrthomas) → review+
Comment on attachment 476840 [details] [diff] [review]
OPSI packages for ssh config/known hosts

changeset:   63:d9a4f7770b6a
Attachment #476840 - Flags: checked-in+
Set to deploy on all 32-bit windows build machines (staging and production).
We're all done here.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.