Closed
Bug 669483
Opened 13 years ago
Closed 13 years ago
automation.py log handling chokes on non-ASCII output
Categories
(Testing :: General, defect)
Testing
General
Tracking
(Not tracked)
RESOLVED
FIXED
mozilla8
People
(Reporter: Gavin, Assigned: Gavin)
References
Details
Attachments
(1 file, 1 obsolete file)
1.51 KB,
patch
|
ted
:
review+
|
Details | Diff | Splinter Review |
I wrote a test that dump()s "\xe9" as part of its output. On my local machine, this results in: Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/logging/__init__.py", line 765, in emit self.stream.write(fs % msg.encode("UTF-8")) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 131: ordinal not in range(128) I still haven't figured out exactly what's going on, but I managed to come up with a patch that "fixes" this for me. Need to look into it more.
Assignee | ||
Comment 1•13 years ago
|
||
Assignee: nobody → gavin.sharp
Status: NEW → ASSIGNED
Assignee | ||
Comment 2•13 years ago
|
||
So from what I can gather the problem is this: - the logging code attempts to encode the passed-in string as UTF-8 (the stream's encoding in this case is UTF-8). The encode() method accepts unicode input, but otherwise falls back to assuming ASCII if the input is not of type "unicode" (which is the case for us). - our output is mostly already UTF-8 encoded, e.g. dump() produces bytes 0xC3 0xA9 for the JS string "\xe9" (though try server runs with an earlier patch did reveal that we also occasionally output invalid UTF-8 sequences, like 0xA0 for an NBSP in ISO-8559-1) So when the logging code attempts to interpret our UTF-8 as ASCII, it chokes. We can't just assume that it's all UTF-8, though, since we also produce ISO-8559-1. Thankfully jhammel points out that decode() in Python 2.3 can deal with this case by just ignoring the invalid sequences and leave them as-is, and since they're ISO-8859-1 they're already effectively "unicode".
Assignee | ||
Comment 3•13 years ago
|
||
Attachment #544075 -
Attachment is obsolete: true
Attachment #544272 -
Flags: review?(ted.mielczarek)
Assignee | ||
Comment 4•13 years ago
|
||
This passed a all-unittests tryserver run without issues: http://tbpl.mozilla.org/?tree=Try&rev=3216385011d4
Comment 5•13 years ago
|
||
Comment on attachment 544272 [details] [diff] [review] patch Review of attachment 544272 [details] [diff] [review]: -----------------------------------------------------------------
Attachment #544272 -
Flags: review?(ted.mielczarek) → review+
Assignee | ||
Comment 6•13 years ago
|
||
http://hg.mozilla.org/integration/fx-team/rev/8e54ffd82a58
Assignee | ||
Comment 7•13 years ago
|
||
http://hg.mozilla.org/mozilla-central/rev/8e54ffd82a58
Flags: in-testsuite-
Target Milestone: --- → mozilla8
Assignee | ||
Updated•13 years ago
|
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•