Closed Bug 1258343 Opened 8 years ago Closed 8 years ago

[mozlog] UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position %bar%: ordinal not in range(128)

Categories

(Testing :: Mozbase, defect)

defect
Not set
normal

Tracking

(firefox48 fixed, firefox49 fixed)

RESOLVED FIXED
mozilla49
Tracking Status
firefox48 --- fixed
firefox49 --- fixed

People

(Reporter: whimboo, Assigned: whimboo)

References

Details

Attachments

(1 file)

Seen this in various Marionette based tests (see bug 1255986) when creating a HTML report via mozlog:

00:33:01     INFO -      super(MarionetteTestResult, self).addFailure(test, err)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\moztest\adapters\unit.py", line 140, in addFailure
00:33:01     INFO -      extra=extra)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\logtypes.py", line 47, in inner
00:33:01     INFO -      return f(self, data)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\structuredlog.py", line 346, in test_end
00:33:01     INFO -      self._log_data("test_end", data)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\structuredlog.py", line 200, in _log_data
00:33:01     INFO -      self._handle_log(log_data)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\structuredlog.py", line 210, in _handle_log
00:33:01     INFO -      handler(data)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\handlers\base.py", line 93, in __call__
00:33:01     INFO -      formatted = self.formatter(data)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\handlers\base.py", line 62, in __call__
00:33:01     INFO -      return self.inner(item)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\reader.py", line 68, in __call__
00:33:01     INFO -      return handler(data)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\formatters\html\html.py", line 94, in test_end
00:33:01     INFO -      self.make_result_html(data)
00:33:01     INFO -    File "C:\slave\test\build\venv\lib\site-packages\mozlog\formatters\html\html.py", line 125, in make_result_html
00:33:01     INFO -      href = 'data:text/plain;charset=utf-8;base64,%s' % base64.b64encode(str(content).encode('utf-8'))
00:33:01    ERROR -  UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position 29763: ordinal not in range(128)
https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozlog/mozlog/formatters/html/html.py#135

> href = 'data:text/plain;charset=utf-8;base64,%s' % base64.b64encode(str(content).encode('utf-8'))

Given that we expect UTF strings here, shouldn't we make the formatting string UTF too?

I feel this could be a good first bug. Joel, would you mentor this bug?
Flags: needinfo?(jmaher)
this seems like a fixable thing, just reading this bug I would have no idea without researching mozlog and string encoding in python on what to do here.  Maybe this wouldn't be a good first bug, but possible a good next bug?
Flags: needinfo?(jmaher)
As said the fix would actually be:

> - href = 'data:text/plain;charset=utf-8;base64,%s' % base64.b64encode(str(content).encode('utf-8'))
> + href = u'data:text/plain;charset=utf-8;base64,%s' % base64.b64encode(str(content).encode('utf-8'))

This forces the formatting string to actually be a unicode one. Testing would still be necessary.
I would be happy to mentor this.

As an interested contributor, there is an example above of what needs to happen.  Your job will be to look through the rest of the code and ensure we are using unicdoe everywhere, verifying with unittests that this works, and writing some unittests to ensure we are covering this case.
Mentor: jmaher
Whiteboard: [good next bug][lang=python]
Blocks: 1253209
Assignee: nobody → hskupin
Mentor: jmaher
Status: NEW → ASSIGNED
Whiteboard: [good next bug][lang=python]
Attachment #8750470 - Flags: review?(jmaher) → review+
Comment on attachment 8750470 [details]
MozReview Request: Bug 1258343 - [mozlog] Fix HTML report generation for unicode characters. r=jmaher

https://reviewboard.mozilla.org/r/51431/#review48371

thanks for the unittest with this!
Blocks: 1271612
https://hg.mozilla.org/mozilla-central/rev/2d3c70bc4656
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla49
Test-only change which we need to be landed on mozilla-aurora to get a regression fixed.
Whiteboard: [checkin-needed-aurora]
You need to log in before you can comment on or make changes to this bug.