Closed Bug 889467 Opened 11 years ago Closed 10 years ago

[tracker] upgrade to django 1.5

Categories

(support.mozilla.org :: Code Quality, task, P2)

Product:
support.mozilla.org
Component:
Code Quality
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
2014Q1

People

(Reporter: willkg, Assigned: rrosario)

References

Details

(Whiteboard: u=dev c=general p=3 s=2014.2) 

Django 1.5 has been out for a while. We should upgrade. Srsly.

Note: It's likely this bug will spin off a bunch of other bugs for upgrading vendor libs and/or swapping them out for other libs.
Here are some possible issues we might run into:

* Does django-cache-machine work with Django 1.5?
* there are a bunch of django-nose pull requests and issues related to django 1.5
* tastypie doesn't work with django 1.5 for richard--we might have issues with sumo, too
Whiteboard: u=dev c=general p= s=2013.backlog
Target Milestone: --- → 2013Q3
I think they fixed the tastypie issue I bumped into with richard. Having said that, I still think we should continue switching to django-rest-framework.
Q3 is over... => Q4
Target Milestone: 2013Q3 → 2013Q4
Ugly. Let me try that again:

Django 1.5 release notes:

https://docs.djangoproject.com/en/dev/releases/1.5/

Interesting bits:

> Management commands do not raise SystemExit any more when called by
> code from call_command. Any exception raised by the command (mostly
> CommandError) is propagated.
>
> Moreover, when you output errors or messages in your custom commands,
> you should now use self.stdout.write('message') and 
> self.stderr.write('error') (see the note on management commands output).


> In the admin, you can now filter users by groups which they are members 
> of.


> The new ALLOWED_HOSTS setting validates the request’s Host header and
> protects against host-poisoning attacks. This setting is now required
> whenever DEBUG is False, or else django.http.HttpRequest.get_host()
> will raise SuspiciousOperation. For more details see the full
> documentation for the new setting.


> The cleaned_data dictionary is now always present after form validation.
> When the form doesn’t validate, it contains only the fields that passed
> validation. You should test the success of the validation with the
> is_valid() method and not with the presence or absence of the
> cleaned_data attribute on the form.
Oops--copied that last comment over from bug #910800 where I had problems with formatting. Ergo the "Ugly. Let me try that again:".

I did some prep for Fjord in https://github.com/mozilla/fjord/pull/182 . That fixes test failures and forms so the site works. Mentioning that here because we'll have similar problems.
Fixed verify_exists usage in PR: https://github.com/mozilla/kitsune/pull/1742

Landed in master in: https://github.com/mozilla/kitsune/commit/0bcf52b


Still a lot to go on this one, but that's one less thing to do.
I just updated Django to 1.5.5 and ran the tests and hit this:

Traceback (most recent call last):
  File "./manage.py", line 47, in <module>
    execute_manager(settings)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/__init__.py", line 469, in execute_manager
    utility.execute()
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/__init__.py", line 272, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/__init__.py", line 77, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/home/willkg/mozilla/kitsune/vendor/src/django-nose/django_nose/management/commands/test.py", line 10, in <module>
    from south.management.commands.test import Command
  File "/home/willkg/mozilla/kitsune/vendor/packages/South/south/management/commands/__init__.py", line 10, in <module>
    import django.template.loaders.app_directories
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/template/loaders/app_directories.py", line 25, in <module>
    raise ImproperlyConfigured('ImportError %s: %s' % (app, e.args[0]))
django.core.exceptions.ImproperlyConfigured: ImportError waffle: cannot import name email_utils


We're going to have to update django-waffle.
Also, we should update the cache settings:

/home/willkg/mozilla/kitsune/vendor/src/django/django/core/cache/backends/memcached.py:146: DeprecationWarning: memcached.CacheClass has been split into memcached.MemcachedCache and memcached.PyLibMCCache. Please update your cache backend setting.
Need to upgrade django-nose and test-utils. PR: https://github.com/mozilla/kitsune/pull/1783
Blocks: 952642
Traceback (most recent call last):
  File "/home/willkg/mozilla/kitsune/kitsune/access/tests/__init__.py", line 23, in setUp
    url = reverse('forums.threads', args=[u'test-forum'])
  File "/home/willkg/mozilla/kitsune/kitsune/sumo/urlresolvers.py", line 51, in reverse
    url = django_reverse(viewname, urlconf, args, kwargs, prefix)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 496, in reverse
    return iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 382, in _reverse_with_prefix
    possibilities = self.reverse_dict.getlist(lookup_view)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 297, in reverse_dict
    self._populate()
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 263, in _populate
    for pattern in reversed(self.url_patterns):
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 347, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 342, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/home/willkg/mozilla/kitsune/kitsune/urls.py", line 42, in <module>
    (r'^kpi/', include('kitsune.kpi.urls')),
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/conf/urls/__init__.py", line 25, in include
    urlconf_module = import_module(urlconf_module)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/home/willkg/mozilla/kitsune/kitsune/kpi/urls.py", line 5, in <module>
    from kitsune.kpi.api import (
  File "/home/willkg/mozilla/kitsune/kitsune/kpi/api.py", line 11, in <module>
    from tastypie.resources import Resource
  File "/home/willkg/mozilla/kitsune/vendor/src/django-tastypie/tastypie/resources.py", line 9, in <module>
    from django.db.models.sql.constants import QUERY_TERMS, LOOKUP_SEP
ImportError: cannot import name LOOKUP_SEP


This might be fixed in a more recent version of tastypie. So we'll have to update or nix it.
Depends on: 956836
PR for tweaking our monkeypatching code to be more explicit and reduce import recursion: https://github.com/mozilla/kitsune/pull/1785

Landed in master in: https://github.com/mozilla/kitsune/commit/8f9ec3c
Turning this into a tracker bug to track all the blocking work so we can work on them individually.
Summary: upgrade to django 1.5 → [tracker] upgrade to django 1.5
Created https://bugzilla.mozilla.org/show_bug.cgi?id=957138 for updating cache settings. I suspect we have to do that *after* we update to Django 1.5 because the new format may not be supported in Django 1.4. Ergo, that bug does not block this one.
Blocks: 950358
Target Milestone: 2013Q4 → 2014Q1
I'm going to pass this back.

It's unblocked from the tastypie issues, so it's time for another "update django in vendor/ and see if the tests pass" iteration.
Target Milestone: 2014Q1 → 2013Q4
Grabbing this and putting into the next sprint for now. If I run into more blockers, I will file them and move this accordingly.
Assignee: nobody → rrosario
Priority: -- → P2
Whiteboard: u=dev c=general p= s=2013.backlog → u=dev c=general p=3 s=2014.1
Target Milestone: 2013Q4 → 2014Q1
oops. next sprint.
Whiteboard: u=dev c=general p=3 s=2014.1 → u=dev c=general p=3 s=2014.2
It's possible this will fix our copy/deepcopy issues in production, too: https://code.djangoproject.com/ticket/20470

I thought there was a bug for that, but I can't seem to find it so I'm mentioning it here.
In a pull request:
https://github.com/mozilla/kitsune/pull/1799

I had to upgrade django-waffle and test_utils.
Deployed to prod now. Looking good so far.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
I'm not seeing any new copy/deepcopy issues, but we're still getting those "object has no attribute" error emails from raven-python trying to repr Options objects.
Depends on: 963124
Blocks: 963124
No longer depends on: 963124
No longer blocks: 963124
Depends on: 963124
Depends on: 963088
Still having copy/deepcopy issues: https://errormill.mozilla.org/support/support/group/154782/

So upgrading Django to 1.5 didn't seem to fix any of the weirdo problems we have.
You need to log in before you can comment on or make changes to this bug.