1173189 - pytz.exceptions:AmbiguousTimeError raised during DST end

Status: RESOLVED WONTFIX

(developer.mozilla.org Graveyard :: Code Cleanup, defect, P2)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

https://developer.mozilla.org/en-US/docs/feeds/atom/tag/Modules throws a 500:

pytz.exceptions:AmbiguousTimeError: 2014-11-02 01:04:12

Traceback (most recent call last):
File "/data/www/developer.mozilla.org/venv/lib/python2.7/site-packages/newrelic-2.50.0.39/newrelic/api/web_transaction.py", line 711, in __iter__
File "/data/www/developer.mozilla.org/venv/lib/python2.7/site-packages/newrelic-2.50.0.39/newrelic/api/web_transaction.py", line 1087, in __call__
File "/data/www/developer.mozilla.org/kuma/vendor/src/django/django/core/handlers/wsgi.py", line 187, in __call__
File "/data/www/developer.mozilla.org/kuma/vendor/src/django/django/core/handlers/base.py", line 111, in get_response
File "/data/www/developer.mozilla.org/venv/lib/python2.7/site-packages/newrelic-2.50.0.39/newrelic/hooks/framework_django.py", line 499, in wrapper
File "/data/www/developer.mozilla.org/kuma/kuma/wiki/feeds.py", line 32, in __call__
File "/data/www/developer.mozilla.org/kuma/vendor/src/django/django/contrib/syndication/views.py", line 44, in __call__
File "/data/www/developer.mozilla.org/kuma/vendor/src/django/django/contrib/syndication/views.py", line 193, in get_feed
File "/data/www/developer.mozilla.org/kuma/vendor/src/django/django/utils/timezone.py", line 358, in make_aware
File "/data/www/developer.mozilla.org/kuma/vendor/packages/pytz/tzinfo.py", line 349, in localize


New Relic: https://rpm.newrelic.com/accounts/263620/applications/3172075/traced_errors/3066ed-d3d2f220-0efd-11e5-a48a-f8bc124250a8

Comment 1

[Jannis Leidel [:jezdez]]

So this is an amazing issue and exactly the type of thing we can now work on now that we're on a sane Django version. In the past we've ignored timezone data completely on Python level and this error shows where it fails. Django tries to interpret certain datetime values that lack timezone info with the timezone we've set for the site but fails in this instance as it can't decide whether it's a daylight saving time or not. This is a legitimate bug and needs to be fixed by looking for the point in our code where we create the datetime values and make them timezone aware.

Comment 2

[Jannis Leidel [:jezdez]]

Which reminds me, part of that work has been done in https://github.com/mozilla/kuma/pull/3113

Comment 3

[John Whitlock [:jwhitlock]]

This is coming up again, because of the DST change in November 2017:

https://sentry.prod.mozaws.net/operations/mdn-prod/issues/222629/activity/

The Django syndication code attempts to change to timezone-aware dates (make_aware), and raises this issue when it calls putz's timezone.localize(value, is_dst=None). We could avoid that by attempting to convert naive publication dates to timezone-aware dates (DocumentsFeed.item_pubdate), which may eliminate once source of this error. This would be less effort than the full conversions to "USE_TZ=True" or PostgreSQL (which I still want to do, but we're resource constrained).

I'm taking this one, because tests need updating to current standards before adding the functionality.

Comment 4

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/7202d2cc4a4612ba661aecae72a446312d035a1a
bug 1173189: Refactor Document feed tests

Convert Document feed tests to use pytest and fixtures. Existing tests
were converted (except for the silly parts), and then additional tests
added to get complete coverage.

https://github.com/mozilla/kuma/commit/e1c7b050f6de4501c06b20d5cb6ed2581b5ce989
bug 1173189: Handle DST transition in feeds

When a naive datetime is ambigious because of the transition from
daylight saving time to standard time, assume that the date is DST. This
avoids a month of feed errors in the second half of the year, until we
can transition to timezone-aware dates.

https://github.com/mozilla/kuma/commit/6c12336a827c52e384e4592b2c0ccb6de24654a4
Merge pull request #4621 from jwhitlock/force-aware-times-1410138

bug 1173189: Force timezone-aware times for feeds

Comment 5

[John Whitlock [:jwhitlock]]

This change has been pushed to staging and production. We'll have to wait until November 2018 to confirm the DST transition isn't an issue. Closing for now.

Comment 6

[John Whitlock [:jwhitlock]]

There's some additional code in attachments (convert_to_utc) that will break with ambiguous dates

https://github.com/mozilla/kuma/pull/4647#discussion_r166734471

Comment 7

[John Whitlock [:jwhitlock]]

DST issues have returned:

https://sentry.prod.mozaws.net/share/issue/7bdef18a68914725b63e00dfd7a531c5/

Comment 8

[John Whitlock [:jwhitlock]]

I'm un-assigning myself, because I'm leaving the MDN team.

The issue is that there is an hour at the end of Daylight Saving Time (DST) where an hour (1 AM for Pacific US) repeats. We store naive times (times without timezone information) in MySQL, and Django raises an AmbiguousTimeError for times during this hour when converting to UTC, needed for many formats. One solution is to assume DST or no DST, which was done in https://github.com/mozilla/kuma/pull/4621 for all date code except for convert_to_utc. This code, used whenever serving an attachment, will raise an exception:

import datetime
from kuma.attachments.utils import convert_to_utc
convert_to_utc(datetime.datetime(2017, 11, 5, 1, 8, 42))

Converting this code would probably be a one-day task. It will start causing errors in November 2019, and it would be nice to fix this before then.

Longer-term solutions would be to centralize all this "convert to timezone-aware" code into one place. This may make it easier to transition to USE_TZ=True.

Comment 9

[BMO Automation]

MDN Web Docs' bug reporting has now moved to GitHub. From now on, please file content bugs at https://github.com/mdn/sprints/issues/ and platform bugs at https://github.com/mdn/kuma/issues/.