Deprecate tinderbox-builds/old directories for desktop & mobile

RESOLVED FIXED

Status

Release Engineering
General
P2
normal
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: nthomas, Assigned: gcox)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments, 1 obsolete attachment)

(Reporter)

Description

3 years ago
Long ago, on a disk filer far far away, we ran out of space for firefox/tinderbox-builds and resorted to cm-ixstore01 to keep older builds (firefox/tinderbox-builds/old). At some point mobile/tinderbox-builds had the same problem.

We have now built a new death star^W^W filer setup with more space, bug 988288, and this is no longer true. We can stop copying around several hundred GB/day over NFS.
(Reporter)

Comment 1

3 years ago
Created attachment 8402410 [details] [diff] [review]
[svn] Step 1

gcox and I are going to move all the content in 
   b2g/tinderbox-builds/old
   firefox/tinderbox-builds/old
   mobile/tinderbox-builds/old
up a directory.

This is the first step:
* stop moving data into old/, but keep expiring data in old/ which is older than 30 days
* add a find to expire data in */tinderbox-builds/, so it doesn't grow without bounds. This won't do anything for nearly a month but better to not forget to add it. It should interact fine with other crons we already have.

Eventually the call to cleanup_tinderbox_builds.sh will go away completely, leaving just the three finds I added here.
Attachment #8402410 - Flags: review?(rail)
Comment on attachment 8402410 [details] [diff] [review]
[svn] Step 1

LGTM.

BTW, xargs rm -rf has been always scary to me. Would it better to add -print0 to find commands and -0 to xargs?
Attachment #8402410 - Flags: review?(rail) → review+
(Reporter)

Comment 3

3 years ago
Comment on attachment 8402410 [details] [diff] [review]
[svn] Step 1

Committed revision 85731.

I'll write another patch to convert all our usage of find+xargs+rm to -print0.
Attachment #8402410 - Flags: checked-in+
(Reporter)

Comment 4

3 years ago
Proceeding as we planned:

Wed Apr 16 13:58:15 PDT 2014
    1G-blocks  Used Available Use% Mounted on
-   9500G     2160G     7341G  23% /mnt/netapp/stage/archive.mozilla.org/pub/firefox/tinderbox-builds
-   7168G     4515G     2654G  63% /mnt/netapp/stage/archive.mozilla.org/pub/firefox/tinderbox-builds/old

gcox, feel free to chop old/ down to 5T in size, it won't be going up again.
(Assignee)

Comment 5

3 years ago
I've been whittling on the 3 old's already (firefox-old was 9t a week ago).  :)
I've left myself a note to watch them, but thanks for the doublecheck.
(Reporter)

Comment 6

3 years ago
gcox, the three old dirs are empty now, feel free to drop the axe on them. I'll go ahead with cleaning up our cron jobs.
(Reporter)

Comment 7

3 years ago
Created attachment 8423598 [details] [diff] [review]
[svn] Step 2

Removes no longer needed cleanup_tinderbox-builds.sh from svn, stop calling it in cron, remove reference in puppet. I'll ask gcox to remove the from manually on upload-cron rather than ensure => absent in puppet.
Attachment #8423598 - Flags: review?(rail)
(Reporter)

Comment 8

3 years ago
Created attachment 8423599 [details] [diff] [review]
[svn] -print0 and cleanup

* find -print0 | xargs -0 rm -rf for xrbld and ffxbld
* consolidate firefox+mobile+b2g tinderbox-builds cleanup to one job, daily
* drops the non-android handling for mobile, we only have android now
* adds mobile/tinderbox-builds/*-l10n cleaning to match firefox
Attachment #8423599 - Flags: review?(rail)
(Assignee)

Comment 9

3 years ago
Comment on attachment 8423599 [details] [diff] [review]
[svn] -print0 and cleanup

Review of attachment 8423599 [details] [diff] [review]:
-----------------------------------------------------------------

Unsolicited nitpick feedback:
xrbld cron, there's a trailing slash in the path of the find statement, ala bug 992680.  While xulrunner doesn't have a submount to cause problems today, it's not doing you any favors, so maybe kill it?
(Assignee)

Comment 10

3 years ago
> gcox, the three old dirs are empty now, feel free to drop the axe on them.
> I'll go ahead with cleaning up our cron jobs.

I'll hold until you're committed.  When I unmount, you'll 'lose' the "old" directories from the tree, and I don't want those jobs to start screaming.  Particularly on a Friday.  :)
(Reporter)

Comment 11

3 years ago
Created attachment 8423618 [details] [diff] [review]
[svn] -print0 and cleanup, v2

Good point. This cleans up trailing slashes (all crontabs), fixes a mistaken '-print 0' in xrbld.
Attachment #8423599 - Attachment is obsolete: true
Attachment #8423599 - Flags: review?(rail)
Attachment #8423618 - Flags: review?(rail)
Attachment #8423598 - Flags: review?(rail) → review+
Attachment #8423618 - Flags: review?(rail) → review+
(Reporter)

Comment 12

3 years ago
Comment on attachment 8423598 [details] [diff] [review]
[svn] Step 2

Deleting       files/bin/cleanup_tinderbox-builds.sh
Sending        files/cron/ffxbld
Sending        manifests/upload_cron.pp
Transmitting file data ..
Committed revision 87635.
Attachment #8423598 - Flags: checked-in+
(Reporter)

Comment 13

3 years ago
Comment on attachment 8423618 [details] [diff] [review]
[svn] -print0 and cleanup, v2

Sending        files/cron/b2gbld
Sending        files/cron/b2gtry
Sending        files/cron/ffxbld
Sending        files/cron/xrbld
Transmitting file data ....
Committed revision 87636.
Attachment #8423618 - Flags: checked-in+
(Reporter)

Comment 14

3 years ago
gcox, over to you. Could you please remove upload-cron:/usr/local/bin/cleanup_tinderbox-builds.sh.
Assignee: nthomas → gcox
(Assignee)

Comment 15

3 years ago
Removed shell script from upload-cron as requested.
unmounted the three 'old' volumes (client-side and filer-side).

Pausing here a couple of days before doing final cleanup of the lingering volume husks.
(Assignee)

Comment 16

3 years ago
'old' vols destroyed.  That should do it.
Thanks for all the reworking and cleanup!
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 17

3 years ago
Woot!
Is there other place than ftp://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/ where I can find older builds ?
(Reporter)

Comment 19

3 years ago
Yes, we keep 24 weeks of tinderbox builds on http://inbound-archive.pub.build.mozilla.org
eg
 http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/

The domain is slightly misleading, as we keep firefox and mobile builds from all branches. That will be fixed up at some point but the url will keep working.
You need to log in before you can comment on or make changes to this bug.