Created attachment 556778 [details] [diff] [review] patch v1 there's a few simple tweaks we can do to the gzip part of the compress_log_file() in processbuild.pl function which should speed it up. first, here's the timings of runs using the same batch of 100 test messages: unpatched: real 0m10.062s user 0m9.386s sys 0m0.164s changing the read buffer from 4k to 64k speeds things up: real 0m8.000s user 0m7.521s sys 0m0.146s and on top of this i figure we may as well change the compression level to Z_BEST_SPEED: real 0m6.241s user 0m5.789s sys 0m0.164s
Attachment #556778 - Flags: review?(bear)
A ~40% improvement in log compression time would be great, but we should check what happens to the disk requirements. We're often space limited on the server, and have shortened the retention period as more branches were added. It'd be a trade-off between processing speed and retention time.
using default compression results in 15m of logs, switching to best_speed increases this to 20m. here's some more timings: comp time size 0 6.32 244m 1 6.30 20m 2 6.38 19m 3 6.38 18m 4 7.30 17m 5 7.43 16m 6 8.12 15m 9 13.7 15m perhaps "4" would be a reasonable mid-ground? i assume getting more disk on the server isn't an option given the short life expectancy of tinderbox.
So bear, how much space is free, and what is the current retention period ? I thought it was as short as 10 or 14 days, but there was a comment recently that it was month.
given the moves away from tinderbox, there isn't much value in resolving this issue.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.