1804437 - Windows build UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4: invalid start byte

Status: RESOLVED WORKSFORME

(Thunderbird :: Build Config, defect)

Description

[Cedric Berte]

Steps to reproduce:

I tried to build on a windows 10 environment and when the internationalization phase happens, I have a bug during the building.

Actual results:

Two things can happen. Either the build is failing with the following error :

1:03.08 93% of entries changed
1:05.27 Checking for missing .ftl files in locale fr
1:07.07 toolkit/locales/update.locale.stub
1:07.41 toolkit/locales/locale.ini.stub
Exception in thread ProcessReaderStdout:
Traceback (most recent call last):
File "C:\app\mozilla-build\python3\lib\threading.py", line 973, in _bootstrap_inner
self.run()
File "C:\app\mozilla-build\python3\lib\threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "d:\jenkins\workspace\dgfip\testjp\sources\testing\mozbase\mozprocess\mozprocess\processhandler.py", line 1176, in _read_stream
line = stream.readline()
File "C:\app\mozilla-build\python3\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4: invalid start byte
1:15.73 707 compiler warnings present.
D:/jenkins/workspace/DGFiP/TestJp/sources/obj-x86_64-w64-mingw32/config.status is out of date with respect to D:/jenkins/workspace/DGFiP/TestJp/sources/obj-x86_64-w64-mingw32/.mozconfig.json
Configure complete!
Be sure to run |mach build| to pick up any changes
Parallelism determined by memory: using 8 jobs for 8 cores based on 32.0 GiB RAM and estimated job size of 1.0 GiB
Single process terminated successfully
Hey! Builds initiated with mach build $A_SPECIFIC_TARGET may not always work, even if the code being built is correct. Consider doing a bare mach build instead.

Or the following error happen and it keeps running and running for hours :

1:03.08 93% of entries changed
1:05.27 Checking for missing .ftl files in locale fr
1:07.07 toolkit/locales/update.locale.stub
1:07.41 toolkit/locales/locale.ini.stub
Exception in thread ProcessReaderStdout:
Traceback (most recent call last):
File "C:\app\mozilla-build\python3\lib\threading.py", line 973, in _bootstrap_inner
self.run()
File "C:\app\mozilla-build\python3\lib\threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "d:\jenkins\workspace\dgfip\testjp\sources\testing\mozbase\mozprocess\mozprocess\processhandler.py", line 1176, in _read_stream
line = stream.readline()
File "C:\app\mozilla-build\python3\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4: invalid start byte

Expected results:

Build is running without error.

Comment 1

[Rob Lemley [:rjl]]

You say "internationalization phase", but it's not clear what command you are running. I'm guessing that you've done something like:

• mach build
• mach package
• made a clone of https://hg.mozilla.org/l10n-central/fr/ someplace(?) what revision?

and the failing command is: ?

• mach build installers-fr

can you try setting MOZPROCESS_DEBUG=1 (environment variable) -- that will hopefully give some more detail as to what's generating this invalid utf-8 character.

A couple other questions...

• What version of MozillaBuild is installed?
• NSIS version (where is it from?)

I'm able to complete installers-fr and langpack-fr using the latest MozillaBuild on Win 10. I'll need more info to triage further.

Comment 2

[Cedric Berte]

It seems that we had some issues with our PATH variable and it was using a bad "make" binary. We cleaned our PATH and the issue disappeared.