783274 - Move old objdirs to C:\OldBuilds on Windows and remove them whole-sale using low-priority I/O

Status: RESOLVED DUPLICATE of bug 712206

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[(no longer active)]

On 08/16/12 09:37 AM, Aryeh Gregor wrote:
> On Thu, Aug 16, 2012 at 4:30 PM, Ben Hearsum <[e-mail]> wrote:
>> I hadn't considered using a background thread to remove it. During
>> pulling/update we're I/O-saturated, I'm not sure about during compile.
>
> In addition to the CPU-bound parts of compile (there surely are some),
> there will be times when any given build machine is entirely idle,
> e.g., off-peak hours when not many people are pushing anything.
> Leaving the remove processes on idle priority will make them more or
> less do nothing until there's no I/O going on.
>
>> Implementing this would be very tricky though....the way the build works
>> is by executing commands serially, so I'm not sure how we'd do this in
>> parallel with compilation. There's probably a way, but we'd have to be
>> reasonably sure it's useful to do before diving deeper, I think.
>
> One simple workaround would be to have the build process just move
> everything to a designated directory, like C:\OldBuilds or whatever.
> You could then have an entirely separate process run in the background
> at idle priority, removing anything it finds in that directory.

Comment 1

[Chris Cooper [:coop] (he/him)]

I'd rather tackle this in the build system itself, but we're also moving cleanup steps out of the build job itself and into pre-flight tasks that run before buildbot even starts (bug 712206).