785410 - Only clobber the current build before starting the build

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect, P3)

Description

[Chris Cooper [:coop] (he/him)]

We needed to re-spin the Windows nightly this morning due to a signing server error. Unfortunately, in the interim someone requested a clobber on a whole bunch of different Windows builds. The check_clobber_times step performed the clobber for all the requested builds *before* starting into the actual nightly build it was running. This added almost 40 minutes before the build even started.

check_clobber_times should definitely check whether the current build needs to be clobbered before starting, but the rest of the clobbers should be done as a final step after any downstream builds have been triggered and before returning the slave to the pool.

We have no way of knowing whether a given build marked for clobber will even run on the same slave, so I think I'd be fine with *not* clobbering the other builds at all until they actually get run on that slave.

Comment 1

[[e-mail] (:bhearsum)]

Will this be solved as part of pre flight tasks?

Comment 2

[Chris Cooper [:coop] (he/him)]

(In reply to Ben Hearsum [:bhearsum] from comment #1)
> Will this be solved as part of pre flight tasks?

It relies on the global clobber check happening pre-flight, but we'd still need to check just the current build as part of the build process. That's either a new build step, or a modification of an existing one.

Comment 3

[Chris AtLee [:catlee]]

Once we do the global clobber pre-flight, we can modify the current in-build clobber to only do the current build directory.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

By the same logic we should change purge_builds.py to only free enough space at the start of the build, and defer any other 'too stale' cleanup to a more appropriate time.

Comment 5

[[e-mail] (:bhearsum)]

Does this matter so much now that we have jacuzzis?

Comment 6

[Chris Cooper [:coop] (he/him)]

(In reply to Ben Hearsum [:bhearsum] from comment #5)
> Does this matter so much now that we have jacuzzis?

Maybe. We've had to disable jacuzzis a few times recently though, so maybe not.