Closed Bug 1483850 Opened 6 years ago Closed 6 years ago

[mozprocess] Intermittent UnicodeEncodeError: 'ascii' codec can't encode characters in position 81-83: ordinal not in range(128)

Categories

(Testing :: Mozbase, defect, P5)

Product:
Testing
Component:
Mozbase
Version:
Version 3
Type:
defect

Tracking

(firefox-esr52 unaffected, firefox-esr60 unaffected, firefox61 unaffected, firefox62 unaffected, firefox63 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla63
Tracking Flags:
Tracking Status
firefox-esr52 --- unaffected
firefox-esr60 --- unaffected
firefox61 --- unaffected
firefox62 --- unaffected
firefox63 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: whimboo)

References

Details

(Keywords: intermittent-failure, regression)

Filed by: hskupin [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=194199539&repo=mozilla-central https://queue.taskcluster.net/v1/task/VmVmQz5NRFaPJUR-uD7J5g/runs/0/artifacts/public/logs/live_backing.log There are a couple of those errors. Here one example: 23:37:39 INFO - Traceback (most recent call last): 23:37:39 INFO - File "c:\mozilla-build\python\Lib\threading.py", line 801, in __bootstrap_inner 23:37:39 INFO - self.run() 23:37:39 INFO - File "c:\mozilla-build\python\Lib\threading.py", line 754, in run 23:37:39 INFO - self.__target(*self.__args, **self.__kwargs) 23:37:39 INFO - File "Z:\task_1534373372\build\venv\lib\site-packages\mozprocess\processhandler.py", line 1031, in _read 23:37:39 INFO - callback(line.rstrip()) 23:37:39 INFO - File "Z:\task_1534373372\build\venv\lib\site-packages\mozprocess\processhandler.py", line 947, in __call__ 23:37:39 INFO - e(*args, **kwargs) 23:37:39 INFO - File "Z:\task_1534373372\build\venv\lib\site-packages\mozprocess\processhandler.py", line 1079, in __call__ 23:37:39 INFO - self.stream.write(line.decode('iso8859-1') + '\n') 23:37:39 INFO - UnicodeEncodeError: 'ascii' codec can't encode characters in position 81-83: ordinal not in range(128)
We should not release the version 1.0.0 before those failures have been fixed.
Flags: needinfo?(dave.hunt)
Pavel: Could you look into writing a failing test for this and see if you can fix it for both Python 2.7 and 3.5? Let me know if you have any questions. We should treat this as a priority so we can release mozprocess v1.0.0.
Flags: needinfo?(dave.hunt) → needinfo?(slepushkin)
1) I think, we are hitting this - https://bugzilla.mozilla.org/show_bug.cgi?id=991866 (RESOLVED INCOMPLETE) 2) I've tried to write a test using cyrillic and simplified Chinese in UTF-8 (in test_process_reader.py) - test worked, error has not reproduced (but I am not on Windows) 3) My first idea for fix would be replace > try: > self.stream.write(line.decode() + '\n') > except UnicodeDecodeError: > # TODO: Workaround for bug #991866 to make sure we can display when > # when normal UTF-8 display is failing > self.stream.write(line.decode('iso8859-1') + '\n') with > self.stream.write(line.decode(errors='replace') + '\n') but people in 991866 rejected it and without test - I don't know why. Don't think this is very helpful ,sorry.
Flags: needinfo?(slepushkin)
Note that Windows is very special when it comes to Unicode. So to reproduce such a problem you should have a Windows machine or VM.
The actual work should now happen on bug 1428713. Marking this regression bug as fixed.
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox63: --- → fixed
Keywords: regression
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
Assignee: nobody → hskupin
You need to log in before you can comment on or make changes to this bug.