400846 - try server win32 builders sometimes hang on the "create upload directory" step

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[[e-mail] (:bhearsum)]

The Windows try server builders sometimes hang when attempting to create the upload directory on build.mozilla.org. This is from the log:
C:\WINDOWS\system32\cmd.exe /c ssh [e-mail] mkdir /builds/tryserver/2007-10-23_06:[e-mail]-1193145383
 in dir E:\builds\sendchange-slave\sendchange-win32\. (timeout 1200 secs)
<snip>
mkdir: cannot create directory `/builds/tryserver/2007-10-23_06:[e-mail]-1193145383': File exists

command timed out: 1200 seconds without output
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1


So, the ssh command does what it is supposed to, but never returns.

Comment 1

[[e-mail] (:bhearsum)]

While working on bug 379278 I discovered that upgrading ssh on msys fixes this problem. I installed the updated ssh on the try server slaves, let's see how it goes.

Comment 2

[[e-mail] (:bhearsum)]

To be clear, these packages were installed from the MSYS Supplementary Tools:
* Openssl 0.9.8e-3
* Zlib 1.2.3
* Minires 1.01-1
* Openssh 4.6p1

They were all downloaded from Sourceforge, http://sourceforge.net/project/showfiles.php?group_id=2435.

The problem hasn't shown up since I installed these but I'm not convinced it's gone yet. I'll leave this open for a week or so, if we don't see the problem at the end of that I'll declare it fixed.

Comment 3

[[e-mail] (:bhearsum)]

No hangs at all since this change was made. I'm pretty confident that this was the source of the problem.

Comment 4

[[e-mail] (:bhearsum)]

Moving try server bugs to the new component.