1135969 - Thunderbird log retention of 24 hours is far too short

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Phil Ringnalda (:philor)]

Doing anything about bug 1118175, so that we'll stop throwing actual failures in there just because they include the non-failure access denied message, requires being able to actually see the logs from Thunderbird jobs, but because we throw them away after 24 hours, unlike the 30 days we keep Firefox logs, so far nobody has ever been quick enough to actually see one.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [svn] 30 day retention for thunderbird/tinderbox-builds

* removes cleanup-hourly-builds, shifting jobs to role accounts
* remove esr24 jobs
* 30 days of tinderbox-builds for thunderbird, and start cleaning up calendar/lightning/tinderbox-builds for the same (currently no clean up)
* seamonkey tinderbox-builds kept for 7 days

Comment 2

[Justin Wood (:Callek)]

Comment on attachment 8568352 [details] [diff] [review]
[svn] 30 day retention for thunderbird/tinderbox-builds

fwiw this looks good to me as someone interested in SeaMonkey/TB here (I was going to comment something like "I don't think SeaMonkey needs 30 days like Firefox, but if we can even get 1 full week that'd be great!) and decided I didn't care enough to distract.

Comment 3

[Greg Cox [:gcox]]

Got an estimate for diskspace needs for this?
These areas do not have split-out tindebox-builds vols on the filer (since they've never needed it), and I'd like to figure out how best to accomodate this, either grow the existing subvols, or quick-split the tinderbox areas to new subvolumes, if they're going to get huuuj.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8568352 [details] [diff] [review]
[svn] 30 day retention for thunderbird/tinderbox-builds

Going to add some l10n cleanup so removing the review request.

Greg, thanks for the reminder, here's a very rough estimate:
- save most of 63G from missing cleanup of calendar/lightning/tinderbox-builds
- save most of 63G from missing cleanup of thunderbird/tinderbox-builds/*-l10n
- one day of everything else is 24G so the simple estimate is 720G for 30 days, but this is highly dependent on the number of pushes today. Today was merge day so it's probably higher than normal. Even so, 720G is still well within the 1.4T which is free right now. I'm happy to do 7 or 14 day retention too, given 1 day has been the norm forever and no-one screamed the house down

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Make that 1.1T free right now.

Comment 6

[Greg Cox [:gcox]]

Given how tbird is already pretty big (the biggest thing outside mobile/b2g/ff), I think I'm going to fragment it in the next TCW for more managability.  I know the S3 move is coming, but it's still pretty hairy behind the scenes, especially if that date slips.

30d tbird, f+ from me.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [svn] 7 day retention for thunderbird/tinderbox-builds

I settled on 7 days for thunderbird/tinderbox-builds. Also adds l10n cleanup for thunderbird and seamonkey.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8568366 [details] [diff] [review]
[svn] 7 day retention for thunderbird/tinderbox-builds

Sending        files/cron/seabld
Sending        files/cron/tbirdbld
Deleting       files/cron.d/cleanup-hourly-builds
Sending        manifests/upload_cron.pp
Transmitting file data ...
Committed revision 101193.

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

I'll monitor cron mail for any issues.