1173395 - [traceback] [celery] Error: Task tidings.events._fire_task: "UnicodeEncodeError('ascii', u'[zh-CN] \\u62d6\\u62fd #815039', 8, 10, 'ordinal not in range(128)')"

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Code Cleanup, defect)

Description

[Luke Crouch [:groovecoder]]

Getting a number of these since the push yesterday:

https://pastebin.mozilla.org/8836407

Comment 1

[Jannis Leidel [:jezdez]]

https://github.com/mozilla/kuma/pull/3265

Comment 2

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/9c7cf516617f572785c5f603df7d00f6353cb46e
Fix bug 1173395 - Don't pass in non-ASCII pseudo filenames to difflib.

This basically works around the upstream issue in difflib, which is documented in the many 3rd party bug reports:
- https://bugzilla.redhat.com/show_bug.cgi?id=1221169
- https://github.com/wbsoft/frescobaldi/issues/674
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=784940
- https://bugzilla.mozilla.org/show_bug.cgi?id=1149667

This also manually encodes the content passed to pytidylib with utf-8 in case something went wrong. See https://github.com/countergram/pytidylib/issues/11 for more info and https://github.com/1flow/python-ftr/commit/ed8015ec9161d86e1b502122b640a1490b546d46 for the inspiration of this work around.

https://github.com/mozilla/kuma/commit/91c2dba0ef597ef40a04ef7ca0673e777730d7b2
Merge pull request #3265 from mozilla/bug1173395

Fix bug 1173395 - Don't pass in non-ASCII pseudo filenames to difflib.

Comment 3

[Luke Crouch [:groovecoder]]

Note: the pull above fixes the ISE from difflib, but we filed https://bugzilla.mozilla.org/show_bug.cgi?id=1175937 for a related ISE from tidylib.