1325787 - After running |./mach bootstrap| post-rust, subconfigure.py fails with "SyntaxError: EOL while scanning string literal"

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Jonathan Watt [:jwatt]]

I just updated my Windows machine to take account of the fact that Rust is now a build prerequisite. I did this by first running rustup-init.exe from https://win.rustup.sh/ manually, but the paths for ~/.cargo/bin weren't picked up. I then tried running |./mach bootstrap|. Building after that resulted in the following error output to the console:

--------------
 0:21.85 creating ./config.data
 0:22.48 Traceback (most recent call last):
 0:22.48   File "c:/Users/jon/mozilla/trees/i/build/subconfigure.py", line 441, in <module>
 0:22.48     sys.exit(main(sys.argv[1:]))
 0:22.48   File "c:/Users/jon/mozilla/trees/i/build/subconfigure.py", line 437, in main
 0:22.48     return prepare(srcdir, objdir, args[3], args[4:])
 0:22.48   File "c:/Users/jon/mozilla/trees/i/build/subconfigure.py", line 211, in prepare
 0:22.48     data = {a: b for [a, b] in eval(input)}
 0:22.48   File "<string>", line 1
 0:22.48     [['env',[['FPS_BROWSER_USER_PROFILE_STRING','Default'],['NODE_PATH','C:/mozilla-build/msys/local/lib/node_modules/'],['FRAMEWORKDIR64','C:\\WINDOWS\\Microsoft.NET\\Framework64'],['MAKE_MODE','unix'],['PROGRAMFILES','C:\\Program Files (x86)'],['SSH_AUTH_SOCK','C:/Users/jon/AppData/Local/Temp/ssh-upcOZc5508/agent.5508'],['LOCALAPPDATA','C:\\Users\\jon\\AppData\\Local'],['PWD','c:/Users/jon/mozilla/trees/obj/i-debug'],['PROGRAMW6432','C:\\Program Files'],['SDKMINORVER','1'],['LOGNAME','jon'],['VS140COMNTOOLS','C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\Common7\\Tools\\'],['INPUTRC','C:/mozilla-build/msys/etc/inputrc'],['WINDOWSSDK_EXECUTABLEPATH_X64','C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v10.0A\\bin\\NETFX 4.6.1 Tools\\x64\\'],['MSYSTEM','MINGW32'],['HOMEPATH','\\'],['WINDOWSSDKDIR','C:\\Program Files (x86)\\Windows Kits\\8.1\\'],['PATH','c:/Users/jon/mozilla/trees/i/memory/jemalloc/helper:/c/Users/jon
 0:22.49                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         ^
 0:22.49 SyntaxError: EOL while scanning string literal
 0:22.51 DEBUG: <truncated - see config.log for full output>
 0:22.51 DEBUG: cl : Command line warning D9002 : ignoring unknown option '-LARGEADDRESSAWARE'
 0:22.51 DEBUG: cl : Command line warning D9002 : ignoring unknown option '-NXCOMPAT'
 0:22.51 DEBUG: cl : Command line warning D9002 : ignoring unknown option '-SAFESEH'
 0:22.51 DEBUG: Microsoft (R) Incremental Linker Version 14.00.24215.1
 0:22.51 DEBUG: Copyright (C) Microsoft Corporation.  All rights reserved.
 0:22.51 DEBUG:
 0:22.51 DEBUG: /out:conftest.exe
 0:22.51 DEBUG: /out:conftest.exe
 0:22.51 DEBUG: conftest.obj
 0:22.51 DEBUG: kernel32.lib
 0:22.51 DEBUG: user32.lib
 0:22.51 DEBUG: gdi32.lib
 0:22.51 DEBUG: winmm.lib
 0:22.51 DEBUG: wsock32.lib
 0:22.51 DEBUG: advapi32.lib
 0:22.51 DEBUG: secur32.lib
 0:22.51 DEBUG: configure:17797: checking for d3d9.h
 0:22.51 DEBUG: configure:17834: checking for d3d10.h
 0:22.51 DEBUG: configure:18878: checking for posix_fadvise
 0:22.51 DEBUG: configure:18878: checking for posix_fallocate
 0:22.51 ERROR: old-configure failed
 0:22.54 *** Fix above errors and then restart with\
 0:22.54                "c:/mozilla-build/mozmake/mozmake.EXE -f client.mk build"
 0:22.54 client.mk:379: recipe for target 'configure' failed
 0:22.54 mozmake.EXE: *** [configure] Error 1
--------------

Comment 1

[Jonathan Watt [:jwatt]]

What happens here is that when I run MozillaBuild's start-shell-msvc2015.bat, that invokes start-shell.bat which invokes %MOZILLABUILD%msys\bin\bash. bash loads %MOZILLABUILD%msys\etc\profile which runs all the scripts in /etc/profile.d/*.sh, which includes profile-rustup.sh. That contains the line:

  WIN_HOME=$(cd "$HOME" && pwd)

then WIN_HOME is added to PATH. Unfortunately for me, I have aliased the 'cd' command to print out the directory that it is changing to. As a result WIN_HOME ends up containing a line break.

I'd note that I've had this alias for many, many years and never encountered a problem before now.

Comment 2

[Jonathan Watt [:jwatt]]

Attachment: patch MozillaBuild

We can fix this easily enough with something like this.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5b9ccdbcb75f
Fix MozillaBuild so profile-rustup.sh doesn't break Windows users who have aliased 'cd'. r=gps

Comment 4

[Mike Hommey [:glandium]]

FWIW, there are two things that strike me here:
- The shebang is wrong (/bash/sh instead of /bin/sh)
- We have the (cd $foo && pwd) thing in plenty of places in configure, why doesn't configure fail for you in the same way?

Comment 5

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/5b9ccdbcb75f