1512492 - Some Android in-log logcats truncated: ordinal not in range

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Testing, defect, P1)

Description

[Geoff Brown [:gbrown]]

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=215767634&repo=try&lineNumber=14763

[task 2018-12-06T17:05:49.793Z] 17:05:49     INFO -  12-06 16:52:46.480 I/Gecko   ( 2421):
[task 2018-12-06T17:05:49.793Z] 17:05:49     INFO -  12-06 16:52:46.480 I/Gecko   ( 2421): {"action":"log","time":1544115166480,"thread":null,"pid":null,"source":"reftest","level":"DEBUG","message":"[CONTENT] OnDocumentLoad triggering WaitForTestEnd"}
[task 2018-12-06T17:05:49.794Z] 17:05:49     INFO -  WARNING: Error getting device information: 'ascii' codec can't encode character u'\u201c' in position 65: ordinal not in range(128)

Comment 1

[Geoff Brown [:gbrown]]

This is likely happening here:

Similar code here:

Comment 2

[Geoff Brown [:gbrown]]

This is likely happening here:

https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/layout/tools/reftest/remotereftest.py#329

Similar code here:

https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/testing/mochitest/runtestsremote.py#286

Comment 3

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #0)
> https://treeherder.mozilla.org/logviewer.html#/
> jobs?job_id=215767634&repo=try&lineNumber=14763
> 
> [task 2018-12-06T17:05:49.793Z] 17:05:49     INFO -  12-06 16:52:46.480
> I/Gecko   ( 2421):
> [task 2018-12-06T17:05:49.793Z] 17:05:49     INFO -  12-06 16:52:46.480
> I/Gecko   ( 2421):
> {"action":"log","time":1544115166480,"thread":null,"pid":null,"source":
> "reftest","level":"DEBUG","message":"[CONTENT] OnDocumentLoad triggering
> WaitForTestEnd"}
> [task 2018-12-06T17:05:49.794Z] 17:05:49     INFO -  WARNING: Error getting
> device information: 'ascii' codec can't encode character u'\u201c' in
> position 65: ordinal not in range(128)

The corresponding line in the logcat artifact is

12-06 16:52:46.500  2491  2506 E Web Content: [JavaScript Error: "The “coords†attribute of the <area shape="rect"> tag is not in the “left,top,right,bottom†format." {file: "http://10.0.2.2:8854/tests/dom/base/crashtests/340733-1.html" line: 0 column: 0 source: "coords="""}]

Note non-ascii characters (quotes?) around "coords", and "left,top,right,bottom"

Comment 4

[Geoff Brown [:gbrown]]

Attachment: decode from logcat, encode before printing

This follows the "unicode sandwich" advice of, say, https://stackoverflow.com/questions/21129020/how-to-fix-unicodedecodeerror-ascii-codec-cant-decode-byte.

It seems to avoid decode failures:

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=053a94afaa6889dced3f3f6e97a855c5f27ddfc1

And continues until the end of the jobs: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=215888544&repo=try&lineNumber=27417

I haven't found specific examples of similar JavaScript errors in this try run, but if I apply the same code manually to data like comment 3, it survives printing and is fairly readable.

Comment 5

[Bob Clary [:bc] (inactive)]

Comment on attachment 9029978 [details] [diff] [review]
decode from logcat, encode before printing

Review of attachment 9029978 [details] [diff] [review]:
-----------------------------------------------------------------

I wish we had a general way to fix this but I'm ok with it as is. r+.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c6c666505b21
Improve support for unicode logcat content in android test logs; r=bc

Comment 7

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/c6c666505b21