633784 - Automate log backup to S3

Status: RESOLVED WONTFIX

(Camino Graveyard :: Product Site, defect)

Description

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

I realized the other day that not only have I not finished uploading the last 3 massive monthly logfiles from the old server, but also that I was accumulating a large collection of weekly logfiles that needed to get backed up to S3.  I'm slowly working through those as I have the spare bandwidth, but it seems like a suboptimal solution.

I think what we ought to do is have the server upload the completed week's file shortly after logrotate finishes rotating it (and renaming it sanely, per 621569).

Specifically:

1) Fix bug 621569
2) Create a new S3 key for the server
3) Have the crontab run a small script, say, about 5 min after log rotation occurs that calls s3cmd with all the appropriate flags and uploads that day's file.

Comment 1

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to comment #0)
> 3) Have the crontab run a small script, say, about 5 min after log rotation
> occurs that calls s3cmd with all the appropriate flags and uploads that day's
> file.

logrotate also supports running a script after it's done.

Comment 2

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

We should also have the server gzip the logs first (I think logrotate can do that?) so that 80 MB -> 6MB before upload.

Comment 3

[Samuel Sidler (:ss)]

Given the current state of the Camino project, we won't be fixing these website bugs. Mass changing our Product Site bugs (search on "camino-website-bugs").

RESOLVED -> WONTFIX.