768698 - db backup scripts need tweaking to prevent high load and disk space

Status: RESOLVED FIXED

(Infrastructure & Operations :: Infrastructure: Other, task)

Description

[Matt Pressman [:mpressman]]

Our current backup job seems to kick off all our db backups at once and one part of the backup, rawcopies just get a tar file created. This has caused high load and critical disk alerts. I think if run one at a time it will prevent the load issues and compressing while archiving the rawcopies like what is done in the sqldumps section will prevent the disk space issues.

Comment 1

[Rob Tucker [:rtucker]]

mpressman,
Would love to help out with this, I don't know where this script is though. I'll need some direction from you on where to start looking.

Comment 2

[Matt Pressman [:mpressman]]

/etc/cron.d/backups

Comment 3

[Matt Pressman [:mpressman]]

on backup1.db.scl3.mozilla.com

Comment 4

[Rob Tucker [:rtucker]]

Took some doing but reverse enginner this, but i figured out how it works. I added to the actual backup script to gzip and removed the extra cron entry that would then go through and gzip them up. This should help tremendously with the disk space issues.

Comment 5

[Rob Tucker [:rtucker]]

I went and took a look this morning. The changes to the script seem to be working perfectly.

mpressman, will you confirm and close this out?

Comment 6

[Matt Pressman [:mpressman]]

great job rtucker!!!