668652 - [amo] The CDN is not gzipping assets

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Jeff Balogh (:jbalogh)]

The CDN is going through static-cdn.addons.mozilla.net and the origin server is static.addons.mozilla.net. The origin returns gzipped content. The CDN does not.

Origin (gzipped):
$ curl -skI -H 'Host: static.addons.mozilla.net' -H 'Accept-encoding: gzip' 'https://63.245.209.91/media/js/impala-min.js?build=dc4ba05' | grep Content-Encoding
Content-Encoding: gzip

CDN (plain text):
$ curl -H'Accept-encoding: gzip' 'https://static-cdn.addons.mozilla.net/media/js/impala-min.js?build=dc4ba05'
<lots of plain text>

Compressing our assets will reduce their transfer size by 253.4KiB (71% reduction).

Comment 1

[Jake Maul [:jakem]]

Fortunately there is a CDN setting for this, and as you've noticed it is currently disabled.

We can enable it, but need to know what MIME types should be compressed. The default list is:

text/plain
text/html
text/css
application/x-javascript
text/javascript

Would you care to add or remove anything? Unfortunately it doesn't seem possible to simply pass on whatever the origin server does...

Comment 2

[Jeff Balogh (:jbalogh)]

Is there a place where we have this in zeus? I don't know the full list, but we should add these too:

application/json
image/svg+xml

Comment 3

[Wil Clouser [:clouserw]]

I thought we enabled the "respect our headers" option so it would pass requests through and cache whatever we gave it.

Is it possible to control this from apache instead of from their interface?  I'm afraid that once we decide on this list we'll all promptly forget what is on it.

Comment 4

[Jake Maul [:jakem]]

The "respect our headers" thing applies only to cache headers... doesn't affect compression. As far as I can see, there's no good way to tell the CDN how act based on how the origin responds.

This kinda makes sense: it's a separate connection between us and Edgecast and Edgecast to the customer. Origin->Edgecast might be compressed, independently of whether Edgecast->customer is. For instance, the customer might be using an older browser that doesn't support compression... but this doesn't mean that Edgecast should request the content from the origin uncompressed... that leg can still be gzipped and at least save us some bandwidth.

This might also allow them to handle only one cache entry for an object, and serve it up either compressed or uncompressed, based on what the client requests. If it just passes through, it must store 2 copies. (I'm not saying how it does it, just rationalizing why they might choose one way over another.)


In any case, it's easy to look up the current setting in the Edgecast control panel, so I'm not worried about it becoming a problem. I've added those 2 to the list and turned it on. Propagation is given as 1 hour.

I'll keep this open until I can see the change.

Comment 5

[Jake Maul [:jakem]]

Seems to be working. YSlow reports things being gzipped:

Type    Original  gzip  URL
js	250.9K	75.4K	https://static-cdn.addons.mozilla.net/media/js/common-min.js?build=dc4ba05

Also that curl job seems to be returning gzipped content for me now. Oddly, the Firefox web console does not show the 'Content-Encoding' header, but curl does... might be getting filtered out of the display or something.