1013865 - Use contrib.staticfiles

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Code Quality, enhancement, P5)

Description

[Mathieu Agopian [:magopian]]

Use django.contrib.staticfiles to deal with statics, instead of our home made solution.

This could also maybe help us use django-pipeline or django-compressor.

Comment 1

[Yohan Boniface [:ybon]]

PR: https://github.com/mozilla/olympia/pull/110

Comment 2

[Yohan Boniface [:ybon]]

Status update on this.

Big picture: we are close to be ready to land, next step is to have as much possible possible to review/comment it, to be sure the landing will be graceful, and then to merge in master and put it live in dev server asap.

A bit of vocabulary:
statics: versionned file bundled with the code base of type css,js,img...
medias: files that are uploaded by users from the site


The goal here is to:
- better separate statics and medias management
- djangoïse AMO a bit more (cut down AMO learning curve)
- simplify url and path settings
- move forward on the way to have a Django project plug and play for new contributors


Overview of the changes:
- `STATIC_URL` and `MEDIA_URL` have been split (`STATIC_URL` default path will be `/static/`, default `MEDIA_URL` will be `/storage`: we have chose not to use `/media/` to avoid confusion, and to make easier to manage redirects after first push)
- `STATIC_ROOT` and `MEDIA_ROOT` have been split
- we will need to decide of a `STATIC_ROOT` for each of our servers (dev, stage, prod)
- we will need to add `collectstatic` command to our deploy process (already added in `fabfile.py` and `update.py`)
- we will need to update our Nginx config to server statics from this `STATIC_ROOT` folder
- versionned static files have been moved to /static instead of /media (but we don't serve them from here in stage/prod)
- we will need to handle redirect from /media/ to /statics
- `MEDIA_ROOT` is the new `UPLOADS_PATH`
- we will need to handle redirect from /img/uploads/* to /storage/*
- persona media are a special case: they were served from a `LOCAL_MIRROR_URL`, and they will be served as normal medias
- all the media related `XXX_PATH` settings have been removed, but can still be used to override default storage path where neede; instead we use the helper `storage_path`
- all the media related `XXX_URL` settings have been removed, but can still be used to override default URL where needed; in the code, we now use the helper `media_url` instead
- the django-moz-header submodules have been also moved in the `media` to `static` move, which make the git game a bit harder, there is a discussion (bug 1035783) about moving them to a real django app
- jingo-minify has been patched (https://github.com/jsocol/jingo-minify/pull/64/files), and the plan in long term is to move over from it (see bug 1024116)



I've created a pad to start preparing the push: https://amo.etherpad.mozilla.org/9

Immediate next step is to have a bit more expert eyes on the PR, thank you all for reading, reviewing, and commenting.

Jason, I add you in needinfo, because I think a sysadmin eye here is specifically needed. I will also ping you on IRC at some point to talk a bit more of the details :)

Comment 3

[Jason Thomas [:jason]]

https://github.com/mozilla-services/svcops-puppet/pull/57

Comment 4

[Yohan Boniface [:ybon]]

Notes for the QA team:

This change has split the management of all the statics (i.e. CSS, javascripts, style images…) and the "user medias", which are basically the content uploaded by the users: addons files, addons icons, themes images, collections icons, etc.

So here are some things to QA:

- check that all the look and feel of the site seems OK
- check that the javascript console has no (new, ahem) error
- check that the upload of an addon is ok and the file is then downloadable
- check that the default addons icons work
- check that we can customize the addons' icons
- check that we can upload some addons previews, and they are not broken on the site
- check that disabled addons cannot be downloaded by anonymous
- check that we can create a theme and its files are uploaded OK and then visible on the site and installable
- check that disabled themes files cannot be viewed nor installed
- check that the icons of the collections are OK, and they can be changed
- check that a user can change it's picture and it's then visible OK on the site

(Note that I'm checking all this on dev myself once :)

Thanks!

Comment 5

[Iulian Timis]

We have verified all the scenarios described above in comment 4 in https://addons.allizom.org/ on FF31 (Win 7) and we have not found any issues regarding the uploaded content.

Comment 6

[Christopher Van Wiemeersch [:cvan]]

(In reply to Iulian Timis from comment #5)
> We have verified all the scenarios described above in comment 4 in
> https://addons.allizom.org/ on FF31 (Win 7) and we have not found any issues
> regarding the uploaded content.

What about https://bugzilla.mozilla.org/show_bug.cgi?id=1023010#c2 ?

Comment 7

[Yohan Boniface [:ybon]]

Yeah, sounds like related, let me check that :)

Comment 8

[Yohan Boniface [:ybon]]

I've answered on bug 1023010

Comment 9

[Iulian Timis]

Verified as fixed based on Comment 5.
Closing bug.