1628621 - Perma win x64 UnicodeEncodeError: 'charmap' codec can't encode characters in position 45-47: character maps to <undefined>

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: archaeopteryx [at] coole-files.de
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=296828736&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/PsHB5lyfRueES6xb-cq_YQ/runs/1/artifacts/public/logs/live_backing.log

---

Since bug 1617748 etc. landed, several Windows x64 debug and asan tasks contain many tracebacks due to encoding error:

[task 2020-04-08T19:31:34.340Z] 19:31:34 INFO - warning: unused doc comment
[task 2020-04-08T19:31:34.341Z] 19:31:34 INFO - --> Z:\task_1586371036\build\src\third_party\rust\unicode-bidi\src\lib.rs:732:9
[task 2020-04-08T19:31:34.341Z] 19:31:34 INFO - |
[task 2020-04-08T19:31:34.341Z] 19:31:34 INFO - 732 | /// Bidi_Class: AL AL AL B L L L
[task 2020-04-08T19:31:34.341Z] 19:31:34 INFO - | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2020-04-08T19:31:34.341Z] 19:31:34 INFO - --- Logging error ---
[task 2020-04-08T19:31:34.341Z] 19:31:34 ERROR - Traceback (most recent call last):
[task 2020-04-08T19:31:34.341Z] 19:31:34 INFO - File "c:\mozilla-build\python3\lib\logging_init_.py", line 994, in emit
[task 2020-04-08T19:31:34.341Z] 19:31:34 INFO - stream.write(msg)
[task 2020-04-08T19:31:34.341Z] 19:31:34 INFO - File "c:\mozilla-build\python3\lib\encodings\cp1252.py", line 19, in encode
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - return codecs.charmap_encode(input,self.errors,encoding_table)[0]
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - UnicodeEncodeError: 'charmap' codec can't encode characters in position 45-47: character maps to <undefined>
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - Call stack:
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - File "c:\mozilla-build\python3\lib\threading.py", line 884, in _bootstrap
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - self._bootstrap_inner()
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - File "c:\mozilla-build\python3\lib\threading.py", line 916, in _bootstrap_inner
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - self.run()
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - File "c:\mozilla-build\python3\lib\threading.py", line 864, in run
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - self._target(*self._args, **self._kwargs)
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - File "Z:\task_1586371036\build\src\testing/mozbase/mozprocess\mozprocess\processhandler.py", line 1116, in _read
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - callback(line.rstrip())
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - File "Z:\task_1586371036\build\src\testing/mozbase/mozprocess\mozprocess\processhandler.py", line 1026, in call
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - e(*args, **kwargs)
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - File "Z:\task_1586371036\build\src\python/mach\mach\mixin\process.py", line 88, in handleLine
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - line_handler(line)
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - File "Z:\task_1586371036\build\src\python/mozbuild\mozbuild\controller\building.py", line 706, in on_line
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - self.log(logging.INFO, 'build_output', {'line': message}, '{line}')
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - File "Z:\task_1586371036\build\src\python/mach\mach\mixin\logging.py", line 54, in log
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - extra={'action': action, 'params': params})
[task 2020-04-08T19:31:34.342Z] 19:31:34 INFO - Message: '{line}'

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Affect task types:
build-win64-asan-fuzzing/opt
build-win64-asan/debug
build-win64-asan/opt
build-win64-fuzzing/debug
build-win64-plain/debug
build-win64-rusttests/debug

Comment 2

[Ricky Stewart]

Attachment: Bug 1628621 - Ensure whenever we write to the terminal, if ascii or charmap are the default locale, use utf-8 instead.

Comment 3

[Intermittent Failures Robot]

3 failures in 4823 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• try: 1

Platform breakdown:

• windows2012-64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1628621&startday=2020-04-06&endday=2020-04-12&tree=all

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/739aa510248d
Ensure whenever we write to the terminal, if ascii or charmap are the default locale, use utf-8 instead. r=glandium

Comment 6

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/739aa510248d

Comment 7

[Intermittent Failures Robot]

2 failures in 4528 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform breakdown:

• windows2012-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1628621&startday=2020-04-13&endday=2020-04-19&tree=all