1164477 - Upgrade to django 1.7

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Code Quality, defect)

Description

[Mathieu Agopian [:magopian]]

It was done on zamboni (https://github.com/mozilla/zamboni/pull/3038) some time ago, we should do it too

Comment 1

[Christopher Grebs [:cgrebs]]

While going through our dependencies to see what needs upgrading I also found a few that might not be used anymore. Is there someone who can say more to that?

protobuf?
thrift?
slumber?
django-filter?
hive? (is Hive being used, who could know that?)
psutil?
sqlparse?
email_reply_parser?
pyasn1?

All of those don't have any references in code anywhere.

Comment 2

[Christopher Grebs [:cgrebs]]

There's a WIP branch at https://github.com/EnTeQuAk/olympia/tree/feature/django1.7 - still looooots of stuff to do.

* piston stands massively in our way
* There are millions of warnings an deprecations (e.g get_query_set vs get_queryset, commit_on_success vs atomic)
* New default test runner warning

But the runserver starts at least, tests fail very hard though (more dependencies that rely on old APIs)

Comment 3

[Mathieu Agopian [:magopian]]

Regarding piston: we'd like to get rid of it (see bug 1217094)

Regarding the dependencies, make sure to check the content of the site-packages (in a venv you can "cdsitepackages" to get dropped right into it)

- protobuf seems to be used by heka
- thrift is used by pyhs2 (the lib to query hive) and maybe elasticsearch (or is it just for some optional backend?)
- slumber: don't think we use that (maybe it was used by the marketplace, and wasn't removed when we split the repository?)
- django-filter is used by DRF (but is optional, not sure if we need it?)
- hive isn't in the requirements files, so if you have it locally, maybe it's because it got installed as a dependency of another requirement? That's odd because we shouldn't be doing that (we use the --no-deps argument to pip). This is worth investigating (possibly using the pip-tools dev branch and its util to build the dependency tree, see what's requiring it)
- psutil not sure why we have that, seems to be used by celery/utils/debug.py (to provide the memsample and memdump utils?)
- sqlparse is used by the django debug toolbar and django-extensions (for runserver_plus and shell_plus). Maybe we can remove those in production, and thus get rid of the sqlparse dependency in production?
- email_reply_parser can go, it was used by the marketplace
- pyasn1 seems to be used by requests, ndg, urllib3

Comment 4

[Christopher Grebs [:cgrebs]]

Thanks for the followup!

* thrift/protobuf, you're right - as discussed on IRC I wasn't aware to what degree we're using heka.
* slumber, email_reply_parser -> great, I'll remove it then. It's nowhere used in our code.

The rest, yeah, seems to be used to some degree. I'll try and see what's going on with the hive requirement.

Comment 5

[Christopher Grebs [:cgrebs]]

Tracking issue on GitHub: https://github.com/mozilla/olympia/issues/905
WIP PR: https://github.com/mozilla/olympia/pull/914

Comment 6

[Christopher Grebs [:cgrebs]]

This is now merged into master, on -dev and will be on -stage next Tuesday.