Closed
Bug 1258700
Opened 9 years ago
Closed 9 years ago
Replace wsgi-sslify with WhiteNoise 3.0's middleware-based approach & use HSTS, Brotli compression
Categories
(Tree Management :: Treeherder, defect, P3)
Tree Management
Treeherder
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: emorley, Assigned: emorley)
Details
Attachments
(1 file, 1 obsolete file)
WhiteNoise 3.0 now uses a Django middleware-based approach of integrating with Django, which means (a) simpler integration, (b) the ability to still use Django's security middleware [1] for things like HTTPS redirection, avoiding the need for yet another package (wsgi-sslify).
It also adds Brotli compression support, improved mimetype handling and more consistent behaviour between development and production environments:
http://whitenoise.evans.io/en/latest/changelog.html
David Evans asked if I could give it a spin before he produced the final release, and from my testing locally it works really well.
[1] https://docs.djangoproject.com/en/1.8/ref/middleware/#django.middleware.security.SecurityMiddleware
Comment hidden (obsolete) |
Comment 2•9 years ago
|
||
Assignee | ||
Updated•9 years ago
|
Attachment #8733415 -
Attachment is obsolete: true
Assignee | ||
Updated•9 years ago
|
Attachment #8733416 -
Flags: review?(cdawson)
Comment 3•9 years ago
|
||
Comment on attachment 8733416 [details] [review]
[treeherder] mozilla:whitenoise-3.0 > mozilla:master
Very thorough! :)
Attachment #8733416 -
Flags: review?(cdawson) → review+
Comment 4•9 years ago
|
||
Commits pushed to master at https://github.com/mozilla/treeherder
https://github.com/mozilla/treeherder/commit/5b48f4b49bb3da8a8155967cf54d35f7f08a8657
Bug 1258700 - Update to WhiteNoise 3.0
There are some backwards incompatible changes:
http://whitenoise.evans.io/en/latest/changelog.html
https://github.com/evansd/whitenoise/compare/v2.0.6...v3.0
Specifically:
* The CLI compression utility must now be called via
`python -m whitenoise.compress` rather than `python -m whitenoise.gzip`.
* The `whitenoise.django.GzipManifestStaticFilesStorage` storage backend
has moved to `whitenoise.storage.CompressedManifestStaticFilesStorage`.
* The internal `add_files()` method has been split into two and the part
which we need to subclass is now named `update_files_dictionary()`. See:
https://github.com/evansd/whitenoise/commit/07f9c0bece4f8b56db253221634f5109294615f3
https://github.com/mozilla/treeherder/commit/25a00716af60753d3632c708bddd907d8bba83b4
Bug 1258700 - Use WhiteNoise as middleware rather than as a WSGI app
WhiteNoise now supports being used as Django middleware, rather than
having to wrap the Django app with the standalone WSGI WhiteNoise app:
http://whitenoise.evans.io/en/latest/changelog.html#simpler-cleaner-django-middleware-integration
http://whitenoise.evans.io/en/latest/django.html#enable-whitenoise
In addition to reducing the complexity of the WGSI configuration, it
means that Django's security middleware can soon be used to perform
HTTPS redirection (and more), rather than having to rely on yet another
(and less featureful) standalone WSGI app: wsgi-sslify.
https://github.com/mozilla/treeherder/commit/23c4a5ea14908bf94dd75374a872c6e4b9e94e76
Bug 1258700 - Generate Brotli-compressed versions of static files
WhiteNoise 3.0 now supports serving Brotli-compressed files to browsers
whose `Accept-Encoding` includes `br`. Note: Both Firefox and Chrome
only support Brotli over HTTPS.
To take advantage of this, the Brotli package just needs to be available
when the compression tool (`python -m whitenoise.compress`) is run. See:
http://whitenoise.evans.io/en/latest/changelog.html#brotli-compression-support
http://whitenoise.evans.io/en/latest/django.html#brotli-compression
The WhiteNoise docs say to use an unofficial PyPI package (brotlipy),
however this has a dependency on libffi (via cffi) and the official repo
now has it's own Python wrapper that does not. As such, this commit
instead uses the official Brotli package from GitHub, whilst we wait for
the official PyPI release (https://github.com/google/brotli/issues/72).
The Brotli install works fine on stage/prod/Heroku/Travis. The Vagrant
environment was missing g++, which is now installed during provision.
https://github.com/mozilla/treeherder/commit/b6ed225ca4c4c1082a217daf29d367c767fd7463
Bug 1258700 - Disable Django's inbuilt static file handling
Since otherwise WhiteNoise will only serve files when using gunicorn
and not when using `./manage.py runserver`, creating inconsistencies
between production and development environments:
http://whitenoise.evans.io/en/latest/changelog.html#use-whitenoise-in-development-with-django
http://whitenoise.evans.io/en/latest/django.html#using-whitenoise-in-development
https://github.com/mozilla/treeherder/commit/44b536205b44200842236b02a1be4cd571e144d0
Bug 1258700 - Add docstrings to CustomWhiteNoise
https://github.com/mozilla/treeherder/commit/05763288fb5e7b973cd428c32c1f3cf29f5b6622
Bug 1258700 - Replace wsgi-sslify with Django's security middleware
Now that we're using WhiteNoise as middleware, Django's security
middleware can be placed in front of it, making the standalone WSGI app
wsgi-sslify redundant, by using `SECURE_SSL_REDIRECT` and
`SECURE_HSTS_SECONDS`:
https://docs.djangoproject.com/en/1.8/ref/middleware/#django.middleware.security.SecurityMiddleware
https://docs.djangoproject.com/en/1.8/ref/settings/#secure-ssl-redirect
https://docs.djangoproject.com/en/1.8/ref/settings/#secure-hsts-seconds
We will also now be able to enable other Django security middleware
features in the future, that were not present in wsgi-sslify (such as
setting the `X-Content-Type-Options` and `X-XSS-Protection` headers).
Werkzerg has been moved to `requirements/dev.txt` since it's now only
needed in development, by django-extension's `runserver_plus` command.
https://github.com/mozilla/treeherder/commit/3d4b43bcaadc857daf20e8c723180a545c58a9da
Bug 1258700 - Enable Django HTTPS redirection/HSTS on stage/prod too
Previously HTTPS redirection was only enabled on Heroku, since
stage/prod handled it on the load balancer. However, the load balancer
isn't setting the HSTS header, and deployment-specific environment
variables (such as `IS_HEROKU`) should really be avoided.
As such, the conditional instead now checks whether `SITE_URL` begins
with `https://`. This has the effect of enabling these Django security
features on stage/prod, but keeping them disabled locally/on Travis,
where the site isn't accessible over HTTPS.
Assignee | ||
Updated•9 years ago
|
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Comment 5•9 years ago
|
||
Commit pushed to master at https://github.com/mozilla/treeherder
https://github.com/mozilla/treeherder/commit/8a0c1ced765f794fbbcab62723235b9e1efdaefd
Bug 1258700 - Make SECURE_HSTS_SECONDS an int not a timedelta
Since otherwise it results in a header of form:
`strict-transport-security: max-age=365 days, 0:00:00`
...rather than:
`strict-transport-security: max-age=31536000`
Assignee | ||
Comment 6•9 years ago
|
||
We now have HSTS headers on !Heroku too :-)
$ curl -I https://treeherder.allizom.org/
HTTP/1.1 200 OK
Date: Wed, 30 Mar 2016 13:01:53 GMT
Server: gunicorn/19.4.5
X-Backend-Server: treeherder2.stage.webapp.scl3.mozilla.com
strict-transport-security: max-age=31536000
Vary: Accept-Encoding
Last-Modified: Wed, 30 Mar 2016 12:57:34 GMT
Cache-Control: public, max-age=60
Access-Control-Allow-Origin: *
Content-Type: text/html; charset="utf-8"
Content-Length: 7386
$ curl -I https://treeherder.allizom.org/static/embed/css/embed.css
HTTP/1.1 200 OK
Date: Wed, 30 Mar 2016 13:01:33 GMT
Server: gunicorn/19.4.5
X-Backend-Server: treeherder2.stage.webapp.scl3.mozilla.com
Content-Length: 181
strict-transport-security: max-age=31536000
Vary: Accept-Encoding
Last-Modified: Wed, 26 Aug 2015 22:02:49 GMT
Cache-Control: public, max-age=60
Access-Control-Allow-Origin: *
Content-Type: text/css; charset="utf-8"
Assignee | ||
Updated•8 years ago
|
Summary: Update to WhiteNoise 3.0 and switch to its middleware-based approach → Replace wsgi-sslify with WhiteNoise 3.0's middleware-based approach & use HSTS, Brotli compression
You need to log in
before you can comment on or make changes to this bug.
Description
•