574524 - Have Apache gzip output of Tinderbox CGIs

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Robert Helmer [:rhelmer]]

Bug 390341 adds a cache for the brief and full reports that tinderbox generates as well as gzip encoding support, but requires a lot of change to Tinderbox and testing which may be still a ways away.

Perhaps the Apache that runs tinderbox.m.o could use mod_deflate and gzip-encode text/html and text/xml instead of the CGI having to do it:

AddOutputFilterByType DEFLATE text/html text/plain text/xml

Please note that this is in addition to the text/javascript change added in bug 571085.

This should probably be tested on tinderbox-stage.m.o first, I have done a little local testing but it should be tested on real data first.

Comment 1

[Shyam Mani [:fox2mike]]

Done. Unfortunately, tinderbox-stage currently points to tinderbox...so it's the same change for both.

Reopen in case you see any issues.

Comment 2

[Shyam Mani [:fox2mike]]

Disregard that last comment, I made the change to stage. Production tinderbox isn't the same and doesn't have this, yet.

Comment 3

[Robert Helmer [:rhelmer]]

Cool thanks this helps a ton... it's a gigantic difference, even though the latter case had a much higher avg dload speed:

"""
$ curl --header 'accept-encoding: gzip' 'http://tinderbox-stage.mozilla.org/showlog.cgi?log=Firefox/1277739863.1277741220.10483.gz' > test.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  809k    0  809k    0     0  27818      0 --:--:--  0:00:29 --:--:-- 24229

$ curl 'http://tinderbox-stage.mozilla.org/showlog.cgi?log=Firefox/1277739863.1277741220.10483.gz' > test.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 55.8M    0 55.8M    0     0  91959      0 --:--:--  0:10:36 --:--:--   99k
"""

I ran this several times and got consistent results. It seemed to take 10-15s for the download to start, which I assume is the time it takes tinderbox-stage to process the original log file, and is going to of course depend on the size and contents of the original log (~59MB in this case).

For reference, downloading the original log file takes around 10-15s (already gzipped and served without processing).

Comment 4

[Robert Helmer [:rhelmer]]

This could use some testing before being deployed to production, maybe pick a day and announce in blogs/newsgroups/etc?

Shyam do you want a separate bug for that?

Comment 5

[Shyam Mani [:fox2mike]]

Yeah, that would be nice. File a new one, reference this bug (in case I'm not around, someone else will know what needs to be changed).