1087248 - [mozlog] Fix encoding for debug in HTML formatter such as page source

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The HTML formatter currently fails to output debug if it contains characters outside of the default encoding. The page source in B2G often contains such characters.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Fix encoding to utf-8. v1.0

This is how we fixed this in the HTML report currently generated by the Marionette test runner. I'm open to other suggestions.

Comment 2

[James Graham [:jgraham]]

Comment on attachment 8509485 [details] [diff] [review]
Fix encoding to utf-8. v1.0

Review of attachment 8509485 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozbase/mozlog/mozlog/structured/formatters/html/html.py
@@ +96,5 @@
>  
>      def make_result_html(self, data):
> +        # change default encoding to avoid encoding problem for page source
> +        reload(sys)
> +        sys.setdefaultencoding('utf-8')

This is terrifying ;)

I think we should try to track down the places that are assuming ascii and actually fox them to be explicit. If you send me a log file that breaks I'll help with this.

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to James Graham [:jgraham] from comment #2)
> I think we should try to track down the places that are assuming ascii and
> actually fox them to be explicit. If you send me a log file that breaks I'll
> help with this.

I was hoping you'd say that. Traceback and page source that caused it coming up!

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Traceback for encoding issues in HTML report

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Page source causing encoding issues

Comment 6

[James Graham [:jgraham]]

It looks like the content argument in base64.b64encode(content) must be a unicode string. If you instead pass in content.encode("utf8") I expect it will work.

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Fix encoding to utf-8. v2.0

(In reply to James Graham [:jgraham] from comment #6)
> It looks like the content argument in base64.b64encode(content) must be a
> unicode string. If you instead pass in content.encode("utf8") I expect it
> will work.

Yep, that works well. Thanks!

Comment 8

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/5ff9458cb808

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/5ff9458cb808