394850 - Build machines should cleanup old files to avoid running out of disk space

Status: RESOLVED INVALID

(Release Engineering :: General, defect, P3)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

The current build process leaves copies of entire product-directory-trees on different machines throughout the release process, without any automated cleanup/recycling process in place. This causes us to run out of disk space on different machines, at different times and causes complicated manual restarts.

Examples of this fallout are: bug#394757, bug#342972, bug#393565, bug#361548.

Its unclear when we should:
- keep a copy, because it would help us debug issues found after-the-fact
- delete a copy, because its identical to what we have elsewhere

(Setting up nagios monitoring to watch for this might be a good idea also)

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Another example is: bug#395622 

Comment 2

[[e-mail] (:bhearsum)]

I think we've got this in place with the new Makefile and syncing to externalStaging (in the staging environment).

John/Rob, what do you guys think?

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to comment #2)
> I think we've got this in place with the new Makefile and syncing to
> externalStaging (in the staging environment).
> 
> John/Rob, what do you guys think?

I think this bug is not specific enough, given the examples in comment 0. I suggest we mark this INVALID or WONTFIX AND file more specific bugs.

Comment 4

[[e-mail] (:bhearsum)]

As best I can tell, everything that matters here has a bug.