Closed
Bug 1258343
Opened 8 years ago
Closed 8 years ago
[mozlog] UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position %bar%: ordinal not in range(128)
Categories
(Testing :: Mozbase, defect)
Testing
Mozbase
Tracking
(firefox48 fixed, firefox49 fixed)
RESOLVED
FIXED
mozilla49
People
(Reporter: whimboo, Assigned: whimboo)
References
Details
Attachments
(1 file)
Seen this in various Marionette based tests (see bug 1255986) when creating a HTML report via mozlog: 00:33:01 INFO - super(MarionetteTestResult, self).addFailure(test, err) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\moztest\adapters\unit.py", line 140, in addFailure 00:33:01 INFO - extra=extra) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\logtypes.py", line 47, in inner 00:33:01 INFO - return f(self, data) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\structuredlog.py", line 346, in test_end 00:33:01 INFO - self._log_data("test_end", data) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\structuredlog.py", line 200, in _log_data 00:33:01 INFO - self._handle_log(log_data) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\structuredlog.py", line 210, in _handle_log 00:33:01 INFO - handler(data) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\handlers\base.py", line 93, in __call__ 00:33:01 INFO - formatted = self.formatter(data) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\handlers\base.py", line 62, in __call__ 00:33:01 INFO - return self.inner(item) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\reader.py", line 68, in __call__ 00:33:01 INFO - return handler(data) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\formatters\html\html.py", line 94, in test_end 00:33:01 INFO - self.make_result_html(data) 00:33:01 INFO - File "C:\slave\test\build\venv\lib\site-packages\mozlog\formatters\html\html.py", line 125, in make_result_html 00:33:01 INFO - href = 'data:text/plain;charset=utf-8;base64,%s' % base64.b64encode(str(content).encode('utf-8')) 00:33:01 ERROR - UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position 29763: ordinal not in range(128)
Assignee | ||
Comment 1•8 years ago
|
||
https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozlog/mozlog/formatters/html/html.py#135 > href = 'data:text/plain;charset=utf-8;base64,%s' % base64.b64encode(str(content).encode('utf-8')) Given that we expect UTF strings here, shouldn't we make the formatting string UTF too? I feel this could be a good first bug. Joel, would you mentor this bug?
Flags: needinfo?(jmaher)
Comment 2•8 years ago
|
||
this seems like a fixable thing, just reading this bug I would have no idea without researching mozlog and string encoding in python on what to do here. Maybe this wouldn't be a good first bug, but possible a good next bug?
Flags: needinfo?(jmaher)
Assignee | ||
Comment 3•8 years ago
|
||
As said the fix would actually be:
> - href = 'data:text/plain;charset=utf-8;base64,%s' % base64.b64encode(str(content).encode('utf-8'))
> + href = u'data:text/plain;charset=utf-8;base64,%s' % base64.b64encode(str(content).encode('utf-8'))
This forces the formatting string to actually be a unicode one. Testing would still be necessary.
Comment 4•8 years ago
|
||
I would be happy to mentor this. As an interested contributor, there is an example above of what needs to happen. Your job will be to look through the rest of the code and ensure we are using unicdoe everywhere, verifying with unittests that this works, and writing some unittests to ensure we are covering this case.
Mentor: jmaher
Whiteboard: [good next bug][lang=python]
Assignee | ||
Updated•8 years ago
|
Assignee: nobody → hskupin
Mentor: jmaher
Status: NEW → ASSIGNED
Whiteboard: [good next bug][lang=python]
Assignee | ||
Updated•8 years ago
|
status-firefox48:
--- → affected
status-firefox49:
--- → affected
Assignee | ||
Comment 5•8 years ago
|
||
Review commit: https://reviewboard.mozilla.org/r/51431/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/51431/
Attachment #8750470 -
Flags: review?(jmaher)
Updated•8 years ago
|
Attachment #8750470 -
Flags: review?(jmaher) → review+
Comment 6•8 years ago
|
||
Comment on attachment 8750470 [details] MozReview Request: Bug 1258343 - [mozlog] Fix HTML report generation for unicode characters. r=jmaher https://reviewboard.mozilla.org/r/51431/#review48371 thanks for the unittest with this!
Comment 8•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/2d3c70bc4656
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla49
Assignee | ||
Comment 9•8 years ago
|
||
Test-only change which we need to be landed on mozilla-aurora to get a regression fixed.
Whiteboard: [checkin-needed-aurora]
Comment 10•8 years ago
|
||
bugherder uplift |
https://hg.mozilla.org/releases/mozilla-aurora/rev/9ba3a7afa711
Whiteboard: [checkin-needed-aurora]
Comment hidden (Intermittent Failures Robot) |
You need to log in
before you can comment on or make changes to this bug.
Description
•