non-ASCII characters in eslint messages get lost
Categories
(Developer Infrastructure :: Lint and Formatting, defect)
Tracking
(Not tracked)
People
(Reporter: heycam, Assigned: ahal)
References
Details
(Whiteboard: [fixed by bug 1580280])
I'm unsure which layer of code is causing this problem. Please help me redirect to an appropriate component!
I did a try run and got some eslint errors. In the Raw Log on treeheder, I see:
[task 2019-07-17T08:37:55.182Z] TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/layout/tools/layout-debug/ui/content/layoutdebug-helper.js:12:58 | Replace `???.createInstance(Ci.nsILayoutDebuggingTools` with `.createInstance(???Ci.nsILayoutDebuggingTools?` (prettier/prettier)
When I run eslint locally, the output I get is:
/z/moz/g/layout/tools/layout-debug/ui/content/layoutdebug-helper.js
12:58 error Replace `⏎··.createInstance(Ci.nsILayoutDebuggingTools` with `.createInstance(⏎··Ci.nsILayoutDebuggingTools⏎` prettier/prettier (eslint)
so it looks like those non-ASCII characters are getting folded down into "?"s.
Updated•5 years ago
|
Comment 1•5 years ago
|
||
Given that ./mach lint
and ./mach eslint
give the right output, I'm suspecting this is something to do with the Taskcluster setup or the builders not handling unicode text.
Hence, moving it across to Taskcluster where we can hopefully get some hints.
Comment 2•5 years ago
|
||
Taskcluster's able to display unicode just fine:
[taskcluster 2019-07-29 12:54:10.845Z] === Task Starting ===
⏎
[taskcluster 2019-07-29 12:54:11.418Z] === Task Finished ===
[taskcluster 2019-07-29 12:54:11.418Z] Successful task run with exit code: 0 completed in 2.627 seconds
It even looks fine in the raw log.
TC just treats this as an encoded bytestream that it feeds to the browser. Something in between mach and TC is causing the issue..
Comment 3•5 years ago
•
|
||
Forgot to follow this so I didn't see the response.
I've just done a test, and it looks like the encoding is not getting set for python. Locally, I see sys.stdout.encoding
as 'UTF-8' but on the builder it is 'None'. The 'None' makes it fallback to outputting ASCII, hence the issue here.
Anyone have any ideas where we need to set this?
Updated•5 years ago
|
Assignee | ||
Comment 5•5 years ago
|
||
I think this will automatically be fixed once bug 1580280 lands (which is getting closer).
But in the meantime changing this to:
out = formatter(result).encode("mbcs" if sys.platform == "win32" else "utf-8", 'replace')
would probably do the trick.
Assignee | ||
Updated•5 years ago
|
Comment 6•5 years ago
|
||
Now that bug 1580280 landed, I've just tested this here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=24125194b7ab39fe6409bdc41dd4c301133e78cf&selectedJob=270039276
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=270039276&repo=try&lineNumber=292
We're now displaying the ⏎
correctly. Hence calling this as fixed by that bug (as it was the most likely one).
Updated•2 years ago
|
Description
•