508260 - win32 try builds occasionally fail their cleanup step with 'no such file or directory' errors

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[[e-mail] (:bhearsum)]

After switching to rmdir in bug 496712 we're seeing win32 try builds occasionally fail with 'no such file or directory' errors. I would that this is an entirely legitimate error, though I'm not sure why it would happen as all of the slaves should have dirs that need cleaning up before the build runs.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

rmdir has an exit code of 1 if the directory doesn't already exist, say if the machine has been clobbered, or is a new slave, or the builder itself is new. Or more likely here, if we changed the builder directories and added rmdir close enough that not all slaves had had a chance to do all builds.

We could wrap the rmdir in a 'if -e mozilla', providing the error code of an actual cleanup error is propagated forward to buildbot.  In the meantime, these slaves have had directories created
  moz2-win32-slave07, 19    win32-unit/mozilla
  moz2-win32-slave05        sendchange-win32-unittest/mozilla
based on build failures on their last 50 builds.

This bug blocks adding more windows builders to the try server, IMO.

Comment 2

[alice nodelman [:alice] [:anode]]

Occurences of this today on win9 and win11.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

try-w32-slave09,11,13 have been fixed up

(In reply to comment #1)
This should refer to try-w32-slave05,07,19

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Testing a fix on staging.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Quick fix

Comment 7

[alice nodelman [:alice] [:anode]]

Comment on attachment 394174 [details] [diff] [review]
Quick fix

The things we do to make windows happy.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 394174 [details] [diff] [review]
Quick fix

http://hg.mozilla.org/build/buildbot-configs/rev/af476b4f14d3

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

Landed 
 http://hg.mozilla.org/build/buildbot-configs/rev/3e196740024a
to apply the same change to the cvs build factory. Carried r+.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Pushed to prod and staging try masters.