1631199 - configure fails on Japanese locale on Windows native

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Masatoshi Kimura [:emk]]

$ ./mach configure
 0:01.72 Clobber not needed.
 0:01.72 Adding make options from e:\m\mozilla-unified\.mozconfig
    AUTOCLOBBER=1
    MOZ_OBJDIR=e:/m/mozilla-unified/obj-x86_64-pc-mingw32
    OBJDIR=e:/m/mozilla-unified/obj-x86_64-pc-mingw32
    FOUND_MOZCONFIG=e:/m/mozilla-unified/.mozconfig
    export FOUND_MOZCONFIG
 0:01.79 d:\mozilla-build\bin\mozmake.EXE -f client.mk -s configure
 0:01.85 cd e:/m/mozilla-unified/obj-x86_64-pc-mingw32
 0:01.87 e:/m/mozilla-unified/configure
 0:02.53 Re-executing in the virtualenv
 0:03.26 Adding configure options from e:\m\mozilla-unified\.mozconfig
 0:03.26   --enable-application=browser
 0:03.27   --enable-crashreporter
 0:03.27   --enable-warnings-as-errors
 0:03.27   --with-libclang-path=c:/Users/kimu/.mozbuild/clang/bin
 0:03.27   --with-clang-path=c:/Users/kimu/.mozbuild/clang/bin/clang.exe
 0:03.27   --enable-clang-plugin
 0:03.27   --enable-geckodriver
 0:03.27   --host=x86_64-pc-mingw32
 0:03.27   --target=x86_64-pc-mingw32
 0:03.27   CC=clang-cl
 0:03.27   LINKER=lld-link
 0:03.27   CXX=clang-cl
 0:03.27   WIN_UCRT_REDIST_DIR=/d/Program Files (x86)/Windows Kits/10/Redist/ucrt/DLLs/x64
 0:03.27   WIN32_REDIST_DIR=/d/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Redist/MSVC/14.15.26706/x64/Microsoft.VC141.CRT
 0:03.27 checking for vcs source checkout... hg
 0:03.33 checking for a shell... D:/mozilla-build/msys/bin/sh.exe
 0:03.60 checking for host system type... x86_64-pc-mingw32
 0:03.88 checking for target system type... x86_64-pc-mingw32
 0:04.37 checking whether cross compiling... no
 0:04.52 checking for Python 3... e:/m/mozilla-unified/obj-x86_64-pc-mingw32/_virtualenvs/init_py3/Scripts/python.exe (3.7.4)
 0:05.12 checking for Python 2... e:/m/mozilla-unified/obj-x86_64-pc-mingw32/_virtualenvs/init/Scripts/python.exe (2.7.16)
 0:05.12 checking for hg... d:/mozilla-build/python/Scripts/hg.exe
 0:08.16 checking for Mercurial version... 5.1.2
 0:09.21 checking for sparse checkout... no
 0:09.22 checking for yasm... d:/mozilla-build/bin/yasm.exe
 0:09.23 checking yasm version... 1.3.0
 0:09.28 checking for the target C compiler... c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe
 0:09.46 checking whether the target C compiler can be used... yes
 0:09.46 checking the target C compiler version... 9.0.1
 0:09.58 checking the target C compiler works... yes
 0:09.58 checking for the target C++ compiler... c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe
 0:09.78 checking whether the target C++ compiler can be used... yes
 0:09.78 checking the target C++ compiler version... 9.0.1
 0:09.91 checking the target C++ compiler works... yes
 0:09.91 checking for the host C compiler... c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe
 0:10.13 checking whether the host C compiler can be used... yes
 0:10.13 checking the host C compiler version... 9.0.1
 0:10.23 checking the host C compiler works... yes
 0:10.23 checking for the host C++ compiler... c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe
 0:10.42 checking whether the host C++ compiler can be used... yes
 0:10.42 checking the host C++ compiler version... 9.0.1
 0:10.52 checking the host C++ compiler works... yes
 0:10.63 checking for 64-bit OS... yes
 0:10.73 checking for Windows SDK... 0x0a00 in 'D:\Program Files (x86)\Windows Kits\10\'
 0:10.73 checking for Universal CRT SDK... 10.0.17763.0 in 'D:\Program Files (x86)\Windows Kits\10\'
 0:10.74 checking for the Debug Interface Access SDK... D:\PROGRA~2\MICROS~2\2017\Community\DIA SDK
 0:10.75 checking for mt... D:/PROGRA~2/WINDOW~1/10/bin/10.0.17763.0/x64/mt.exe
 0:10.78 checking whether MT is really Microsoft Manifest Tool... yes
 0:10.78 checking for linker... c:/Users/kimu/.mozbuild/clang/bin/lld-link.exe
 0:10.78 checking for host_linker... c:/Users/kimu/.mozbuild/clang/bin/lld-link.exe
 0:10.79 checking for makecab... c:/WINDOWS/System32/makecab.exe
 0:10.79 checking for nasm... c:/Users/kimu/.mozbuild/nasm/nasm.exe
 0:10.81 checking nasm version... 2.14.02
 0:10.81 checking for the assembler... D:/PROGRA~2/MICROS~2/2017/Community/VC/Tools/MSVC/14.15.26726/bin/HostX64/x64/ml64.exe
 0:10.82 checking for rc... D:/PROGRA~2/WINDOW~1/10/bin/10.0.17763.0/x64/rc.exe
 0:10.82 checking for ar... c:/Users/kimu/.mozbuild/clang/bin/llvm-lib.exe
 0:10.84 checking for pkg_config... not found
 0:10.96 checking for stdint.h... yes
 0:11.08 checking for inttypes.h... yes
 0:11.19 checking for malloc.h... yes
 0:11.31 checking for alloca.h... no
 0:11.43 checking for sys/byteorder.h... no
 0:11.55 checking for getopt.h... no
 0:11.65 checking for unistd.h... no
 0:11.76 checking for nl_types.h... no
 0:11.87 checking for cpuid.h... yes
 0:11.97 checking for fts.h... no
 0:12.08 checking for sys/statvfs.h... no
 0:12.18 checking for sys/statfs.h... no
 0:12.28 checking for sys/vfs.h... no
 0:12.38 checking for sys/mount.h... no
 0:12.48 checking for sys/quota.h... no
 0:12.58 checking for sys/queue.h... no
 0:12.69 checking for sys/types.h... yes
 0:12.79 checking for netinet/in.h... no
 0:12.89 checking for byteswap.h... no
 0:12.99 checking for perf_event_open system call... no
 0:13.02 checking for llvm_profdata... c:/Users/kimu/.mozbuild/clang/bin/llvm-profdata.exe
 0:13.06 checking for rustc... c:/Users/kimu/.cargo/bin/rustc.exe
 0:13.09 checking for cargo... c:/Users/kimu/.cargo/bin/cargo.exe
 0:13.18 Actually using 'C:\Users\kimu\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin\rustc.exe'
 0:13.27 Actually using 'C:\Users\kimu\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin\cargo.exe'
 0:13.30 checking rustc version... 1.41.1
 0:13.34 checking cargo version... 1.41.0
 0:13.50 checking for rust target triplet... x86_64-pc-windows-msvc
 0:13.60 checking for rust host triplet... x86_64-pc-windows-msvc
 0:13.63 checking for rustdoc... c:/Users/kimu/.cargo/bin/rustdoc.exe
 0:13.68 checking for cbindgen... c:/Users/kimu/.mozbuild/cbindgen/cbindgen.exe
 0:13.70 checking for rustfmt... c:/Users/kimu/.cargo/bin/rustfmt.exe
 0:13.90 checking for clang for bindgen... c:/Users/kimu/.mozbuild/clang/bin/clang.exe
 0:13.90 checking for libclang for bindgen... c:/Users/kimu/.mozbuild/clang/bin/libclang.dll
 0:13.90 checking that libclang is new enough... yes
 0:13.90 checking bindgen cflags... -x c++ -fno-sized-deallocation -fno-aligned-new -DTRACING=1 -DIMPL_LIBXUL -DMOZILLA_INTERNAL_API -DRUST_BINDGEN -DOS_WIN=1 -DWIN32=1 -D_CRT_USE_BUILTIN_OFFSETOF -DHAVE_VISIBILITY_HIDDEN_ATTRIBUTE=1 -std=gnu++17
 0:13.96 checking for nodejs... c:\Users\kimu\.mozbuild\node\node.EXE (10.19.0)
 0:14.00 checking for tar... D:/mozilla-build/msys/bin/tar.exe
 0:14.01 checking for unzip... d:/mozilla-build/bin/unzip.exe
 0:14.01 checking for zip... d:/mozilla-build/bin/zip.exe
 0:14.03 checking for gn... not found
 0:14.03 checking for the Mozilla API key... no
 0:14.03 checking for the Google Location Service API key... no
 0:14.03 checking for the Google Safebrowsing API key... no
 0:14.03 checking for the Bing API key... no
 0:14.03 checking for the Adjust SDK key... no
 0:14.03 checking for the Leanplum SDK key... no
 0:14.03 checking for the Pocket API key... no
 0:14.04 checking for midl... D:/PROGRA~2/WINDOW~1/10/bin/10.0.17763.0/x64/midl.exe
 0:14.04 checking for llvm-dlltool... c:/Users/kimu/.mozbuild/clang/bin/llvm-dlltool.exe
 0:14.04 checking for fxc... D:/PROGRA~2/WINDOW~1/10/bin/10.0.17763.0/x64/fxc.exe
 0:14.05 checking for D3D compiler DLL... D:\Program Files (x86)\Windows Kits\10\Redist\D3D\x64\d3dcompiler_47.dll
 0:14.05 checking for dump_syms... c:/Users/kimu/.mozbuild/dump_syms/dump_syms.exe
 0:14.05 checking for pdbstr... not found
 0:14.06 checking for winchecksec... not found
 0:14.07 checking for awk... D:/mozilla-build/msys/bin/gawk.exe
 0:14.10 checking for perl... D:/mozilla-build/msys/bin/perl.exe
 0:14.13 checking for minimum required perl version >= 5.006... 5.008008
 0:14.17 checking for full perl installation... yes
 0:14.18 checking for gmake... d:/mozilla-build/bin/mozmake.EXE
 0:14.20 checking for watchman... d:/mozilla-build/bin/watchman.exe
 0:14.21 checking for watchman version... 4.9.1
 0:14.21 checking for watchman Mercurial integration... yes
 0:14.22 checking for xargs... D:/mozilla-build/msys/bin/xargs.exe
 0:14.23 checking for makensisu... d:/mozilla-build/nsis-3.01/makensis-3.01.exe
 0:14.28 checking for NSIS version... 3.01
 0:14.28 checking for 32-bit NSIS... yes
 0:14.29 checking for 7z... d:/mozilla-build/bin/7z.exe
 0:14.29 checking for upx... d:/mozilla-build/bin/upx.exe
 0:14.37 checking for llvm-objdump... c:/Users/kimu/.mozbuild/clang/bin/llvm-objdump.exe
 0:14.37 checking for autoconf... D:/mozilla-build/msys/local/bin/autoconf-2.13
 0:30.16 loading cache ./config.cache
 0:31.15 checking host system type... x86_64-pc-mingw32
 0:31.63 checking target system type... x86_64-pc-mingw32
 0:32.12 checking build system type... x86_64-pc-mingw32
 0:32.37 checking for gcc... (cached) c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe -Xclang -std=gnu99
 0:32.39 checking whether the C compiler (c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe -Xclang -std=gnu99   ) works... (cached) yes
 0:32.39 checking whether the C compiler (c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe -Xclang -std=gnu99   ) is a cross-compiler... no
 0:32.41 checking whether we are using GNU C... (cached) no
 0:32.43 checking whether c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe -Xclang -std=gnu99 accepts -g... (cached) no
 0:34.49 checking for c++... (cached) c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe -Xclang -std=c++17
 0:34.51 checking whether the C++ compiler (c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe -Xclang -std=c++17   ) works... (cached) yes
 0:34.51 checking whether the C++ compiler (c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe -Xclang -std=c++17   ) is a cross-compiler... no
 0:34.53 checking whether we are using GNU C++... (cached) no
 0:34.56 checking whether c:/Users/kimu/.mozbuild/clang/bin/clang-cl.exe -Xclang -std=c++17 accepts -g... (cached) no
 0:34.57 checking for ranlib... (cached) :
 0:34.59 checking for D:/PROGRA~2/MICROS~2/2017/Community/VC/Tools/MSVC/14.15.26726/bin/HostX64/x64/ml64.exe... (cached) D:/PROGRA~2/MICROS~2/2017/Community/VC/Tools/MSVC/14.15.26726/bin/HostX64/x64/ml64.exe
 0:34.62 checking for strip... no
 0:34.66 checking for otool... no
 0:36.63 checking for X... (cached) no
 0:38.23 checking for malloc.h... (cached) yes
 0:38.41 checking for strndup... (cached) no
 0:38.44 checking for posix_memalign... (cached) no
 0:38.48 checking for memalign... (cached) no
 0:38.51 checking for malloc_usable_size... (cached) no
 0:38.74 checking whether malloc_usable_size definition can use const argument... yes
 0:39.09 checking for valloc in malloc.h... no
 0:39.33 checking for valloc in unistd.h... no
 0:39.71 checking for _aligned_malloc in malloc.h... yes
 0:40.28 checking NSPR selection... source-tree
 0:40.54 checking if app-specific confvars.sh exists... e:/m/mozilla-unified/browser/confvars.sh
 0:41.46 checking for wget... (cached) wget
 0:41.76 checking for fdatasync... (cached) no
 0:42.06 checking for valid C compiler optimization flags... yes
 0:42.43 checking for llvm-config... c:\Users\kimu\.mozbuild\clang\bin\llvm-config
 0:45.00 checking what kind of list files are supported by the linker... list
 0:45.60 checking for d3d10.h... (cached) yes
 0:47.48 checking for posix_fadvise... (cached) no
 0:47.51 checking for posix_fallocate... (cached) no
 0:48.10 creating ./config.data
 0:48.77 Creating config.status
 0:48.98 Reticulating splines...
 0:52.80  0:03.84 File already read. Skipping: e:/m/mozilla-unified/gfx/angle/targets/angle_common/moz.build
 0:52.85  0:03.90 File already read. Skipping: e:/m/mozilla-unified/gfx/angle/targets/angle_common/moz.build
 0:52.86  0:03.91 File already read. Skipping: e:/m/mozilla-unified/gfx/angle/targets/angle_common/moz.build
 0:52.87  0:03.91 File already read. Skipping: e:/m/mozilla-unified/gfx/angle/targets/angle_common/moz.build
 0:52.87  0:03.91 File already read. Skipping: e:/m/mozilla-unified/gfx/angle/targets/translator/moz.build
 0:52.87  0:03.91 File already read. Skipping: e:/m/mozilla-unified/gfx/angle/targets/libGLESv2/moz.build
 1:11.35 Traceback (most recent call last):
 1:11.35   File "e:/m/mozilla-unified/configure.py", line 181, in <module>
 1:11.35     sys.exit(main(sys.argv))
 1:11.35   File "e:/m/mozilla-unified/configure.py", line 57, in main
 1:11.35     return config_status(config)
 1:11.35   File "e:/m/mozilla-unified/configure.py", line 176, in config_status
 1:11.35     return config_status(args=[], **normalize(sanitized_config))
 1:11.35   File "e:\m\mozilla-unified\python\mozbuild\mozbuild\config_status.py", line 148, in config_status
 1:11.35     the_backend.consume(definitions)
 1:11.35   File "e:\m\mozilla-unified\python\mozbuild\mozbuild\backend\base.py", line 153, in consume
 1:11.35     old_content = existing.read()
 1:11.35 UnicodeDecodeError: 'cp932' codec can't decode byte 0xef in position 0: illegal multibyte sequence
 1:12.29 *** Fix above errors and then restart with\
 1:12.29                "./mach build"
 1:12.34 mozmake.EXE: *** [client.mk;115: configure] Error 1

Comment 1

[Masatoshi Kimura [:emk]]

Regressed by:
https://hg.mozilla.org/mozilla-central/rev/0ae4e20c74b2550105ec472fc9e52e3aeac2509f

Comment 2

[Kevin Jacobs [:kjacobs]]

This looks like a build issue unrelated to NSS, and unfortunately I don't know how to configure the locale to try to reproduce.

Comment 3

[Masatoshi Kimura [:emk]]

base.py trys reading $objdir\msvc\projects\library_hw-acc-crypto.vcxproj and raises an exception because the file contains a BOM (EF BB BF) that is invalid as a shift_jis (or cp932) sequence.

Comment 4

[Masatoshi Kimura [:emk]]

Attachment: Bug 1631199 - Specify the encoding explicitly when opening backend files to be deleted. r?#build

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f7d6ffc88f12
Specify the encoding explicitly when opening backend files to be deleted. r=firefox-build-system-reviewers,rstewart

Comment 6

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/f7d6ffc88f12