1176489 - Ensure all Django request responses are gzipped

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P3)

Description

[Ed Morley [:emorley]]

From:
https://devcenter.heroku.com/articles/http-routing#gzipped-responses

"Since requests to Cedar apps are made directly to the application server – not proxied through an HTTP server like nginx – any compression of responses must be done within your application."

Now for our static assets, WhiteNoise will handle this (as part of bug 1176482), however for requests that must be served by Django, we need to gzip ourselves - since on Heroku we won't have Zeus doing this for us.

eg:
https://docs.djangoproject.com/en/1.7/ref/middleware/#module-django.middleware.gzip

Comment 1

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #0)
> Now for our static assets, WhiteNoise will handle this (as part of bug
> 1176482), however for requests that must be served by Django, we need to
> gzip ourselves - since on Heroku we won't have Zeus doing this for us.

Clarification: gzipping static assets did not happen as part of bug 1176482 after all, we now have the separate bug 1197796 for that.

Comment 2

[Ed Morley [:emorley]]

We'll need to add 'django.middleware.gzip.GZipMiddleware' to MIDDLEWARE_CLASSES, per:
https://docs.djangoproject.com/en/1.8/ref/middleware/#module-django.middleware.gzip

However since we use Django rest framework extensions etag functions, we'll possible need to create another custom middleware layer that strips the etag ';gzip' prefix, per:
http://chibisov.github.io/drf-extensions/docs/#gzipped-etags

As for ordering of MIDDLEWARE_CLASSES, there's some advice in:
https://docs.djangoproject.com/en/1.8/ref/middleware/#middleware-ordering

Comment 3

[Ed Morley [:emorley]]

We've stopped using the rest framework extension's etag feature, so comment 2 doesn't apply. Reference the BREACH attack mentions on https://docs.djangoproject.com/en/1.8/ref/middleware/#module-django.middleware.gzip , it's somewhat orthogonal to this bug since the issue already exists on stage/prod since they gzip too (and the attack mode still applies regardless of where the compression occurs). Notably there's a ticket and PR open (with activity in the last two weeks) that attempts to mitigate BREACH style attacks:
https://code.djangoproject.com/ticket/20869
https://github.com/django/django/pull/5605

Comment 4

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:enable-gzip-middleware > mozilla:master

Comment 5

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/905e1e4210a4ec484e411391426905202abc0262
Bug 1176489 - Use Django's GZip middleware

The Heroku router doesn't support gzipping content itself, so we have to
do so in the application. WhiteNoise handles serving gzipped static
content already, but this change means that content served by Django
will now be compressed by Django's gzip middleware:
https://docs.djangoproject.com/en/1.8/ref/middleware/#module-django.middleware.gzip