Closed
Bug 508406
Opened 15 years ago
Closed 15 years ago
Speed up backupsnip
Categories
(Release Engineering :: General, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: nthomas, Assigned: catlee)
References
Details
Attachments
(3 files)
986 bytes,
patch
|
nthomas
:
checked-in+
|
Details | Diff | Splinter Review |
1.33 KB,
patch
|
nthomas
:
review+
catlee
:
checked-in+
|
Details | Diff | Splinter Review |
987 bytes,
text/plain
|
nthomas
:
review+
catlee
:
checked-in+
|
Details |
Right now we backup the whole (releases) snippet store each time we want to push out more release snippets, but it takes around 50 minutes to pull tens of thousands of small files in a deep dir structure from the NFS mount. We should investigate doing this smarter, eg making a daily full backup and teaching backupsnip to only backup the directories/snippets that are going to change. This should be muuuuuch master, and may allow us to enforce that backupsnip is run before every snippet push (eg 3.5.2 followed by 3.0.13 followed by 3.0.13 MU).
Updated•15 years ago
|
Component: Release Engineering → Release Engineering: Future
Reporter | ||
Updated•15 years ago
|
Assignee: nobody → nthomas
Whiteboard: Q4
Reporter | ||
Comment 1•15 years ago
|
||
In addition to the smart-backup strategy of only backing up things we're changing, we could also start excluding old snippets. For example, the most recent backup has 232968 directories and files, and 56% of those are for the discontinued versions - Firefox 1.5 & 2.0, plus Thunderbird 1.5 (most of that is from the Fx2 partner nightmare). So there's a better than 2x speed up just from making a one-off backup and then excluding those directories in backupsnip. Comments ?
Comment 2•15 years ago
|
||
(In reply to comment #1)
> In addition to the smart-backup strategy of only backing up things we're
> changing, we could also start excluding old snippets. For example, the most
> recent backup has 232968 directories and files, and 56% of those are for the
> discontinued versions - Firefox 1.5 & 2.0, plus Thunderbird 1.5 (most of that
> is from the Fx2 partner nightmare). So there's a better than 2x speed up just
> from making a one-off backup and then excluding those directories in
> backupsnip. Comments ?
r=bhearsum on excluding snippets for any unsupported products
Reporter | ||
Comment 3•15 years ago
|
||
The old snippets are still present, and are backed up at
/opt/aus2/snippets/backup/20091129-FX15-FX20-TB15-FINAL-BACKUP.tar.bz2
but we won't include them in any future backupsnip's. I tried defining EXCLUDES="--exclude='Firefox/1.5* ..." but the shell was breaking the quoting so I gave it away. Also adds 'time' since we do that frequently.
Checking in backupsnip;
/cvsroot/mozilla/tools/release/bin/backupsnip,v <-- backupsnip
new revision: 1.3; previous revision: 1.2
done
aus2-staging and staging-stage updated.
Attachment #415087 -
Flags: checked-in+
Reporter | ||
Comment 4•15 years ago
|
||
37 minutes for a backupsnip run today, although it does depend a bit on how busy the nfs share is.
Reporter | ||
Comment 5•15 years ago
|
||
Status: some speedup achieved. There are basically three big chunks of data there now - Fx3.0, Fx3.5, and Tb2.0. The approach of comment #0 would maybe get a factor of 3 improvement.
Assignee: nrthomas → nobody
Whiteboard: Q4
Assignee | ||
Updated•15 years ago
|
Assignee: nobody → catlee
Component: Release Engineering: Future → Release Engineering
Assignee | ||
Comment 6•15 years ago
|
||
This patch generates a list of all directories at depth 3 in the staging directory. This corresponds to the '20100115-Firefox-3.6rc2/Firefox/3.6b1/WINNT_x86-msvc', etc. directories.
Directories that don't exist in the live snippets directory are filtered out of this list.
We then tar up all these directories from the live snippets directory.
Switched to using gz instead of bz2 since it's faster as well.
runtime is a few minutes for 3.6 snippets, and 8 minutes for 3.5 snippets using this method. We'd have to generate daily backups, and document recovery procedure as well.
Attachment #422236 -
Flags: review?(nrthomas)
Reporter | ||
Comment 7•15 years ago
|
||
Comment on attachment 422236 [details] [diff] [review]
only backup directories that have changed
Looks good. We'll not save modification timestamps on Firefox/ and Firefox/$version but that seems unlikely to bite us in a recovery situation, which I imagine will be disable all/some updates, restore nightly backup somewhere, apply any updates you want to keep, swap out live directory.
Attachment #422236 -
Flags: review?(nrthomas) → review+
Assignee | ||
Comment 8•15 years ago
|
||
Attachment #422341 -
Flags: review?(nrthomas)
Assignee | ||
Updated•15 years ago
|
Attachment #422341 -
Attachment mime type: application/octet-stream → text/plain
Reporter | ||
Comment 9•15 years ago
|
||
Comment on attachment 422341 [details]
Nightly backup script
Did you decide to stick with bz2 here ?
Attachment #422341 -
Flags: review?(nrthomas) → review+
Assignee | ||
Comment 10•15 years ago
|
||
(In reply to comment #9)
> (From update of attachment 422341 [details])
> Did you decide to stick with bz2 here ?
Yes, the speed here isn't as critical, and we get a small space savings.
Assignee | ||
Comment 11•15 years ago
|
||
Comment on attachment 422236 [details] [diff] [review]
only backup directories that have changed
Checking in backupsnip;
/cvsroot/mozilla/tools/release/bin/backupsnip,v <-- backupsnip
new revision: 1.4; previous revision: 1.3
done
Attachment #422236 -
Flags: checked-in+
Assignee | ||
Comment 12•15 years ago
|
||
Comment on attachment 422341 [details]
Nightly backup script
RCS file: /cvsroot/mozilla/tools/release/bin/backupsnip-nightly,v
done
Checking in backupsnip-nightly;
/cvsroot/mozilla/tools/release/bin/backupsnip-nightly,v <-- backupsnip-nightly
initial revision: 1.1
done
Attachment #422341 -
Flags: checked-in+
Assignee | ||
Comment 13•15 years ago
|
||
backupsnip for 3.5.8 took 17 minutes compared to 42 minutes for 3.5.7.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
You need to log in
before you can comment on or make changes to this bug.
Description
•