1448349 - configure fails when git for Windows is in PATH ahead of MozillaBuild's msys (/usr/bin/grep: nul: No such file or directory)

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Dale Harvey (:daleharvey)]

Fresh build on windows 10, latest mozillaBuild, I have hit this twice in a row following the installation instructions @ https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Windows_Prerequisites

https://gist.github.com/daleharvey/f2d48c0b7b371a22bcbb3469925ca743
is the full log: 

 0:08.67 checking for autoconf... C:/mozilla-build/msys/local/bin/autoconf-2.13
 0:08.67 Refreshing c:/Users/daleharvey/src/gecko-dev/old-configure with C:/mozilla-build/msys/local/bin/autoconf-2.13
 0:08.73 /usr/bin/sed: -e expression #1, char 10: unterminated `s' command
 0:08.85 autoconf: Undefined macros:
0:08.91 /usr/bin/grep: nul: No such file or directory

Seemm the most relevant for the error

Comment 1

[Dale Harvey (:daleharvey)]

So the error here was, this is a git clone, the mozillabuild shell could not see git (although the command prompt could) and failed so I added the git path to my bash_profile, but adding it to the start of the path sonds like its msys tools were used instead of the supported versions causing problems, If I added the git path to the end of my $PATH it looks to be working properly

Comment 2

[(not currently active) Ted Mielczarek]

This sounds like something we could potentially detect in moz.configure. I think the specific error here is probably that the shell we wound up using to execute autoconf was from git. If we're on Windows we could check that the shell in PATH is from MozillaBuild, and error if it's not.

Comment 3

[Gregory Szorc [:gps]]

FWIW `nul` is a reserved filename on Windows. Any attempts to access files of that name will fail.

Git doesn't recognize these reserved filenames and you'll get run-time failures if a file with a reserved filename is in the repo. Mercurial aborts more gracefully.

Not sure where this `nul` file is coming from though.

Comment 4

[Māris Fogels [:mars] (please needinfo)]

Attachment: Bug 1448349 - Detect if autoconf will use the wrong shell on win32 r=#build

Autoconf on Windows will use the first sh it detects in $PATH. If the user has
installed other mingw programs and put them in $PATH before the mozilla-build
tools then Autoconf will pick up a foreign tool's copy of sh. When Autoconf
runs our checks with the foreign sh all of our checks fail.

This patch adds a check to the old.configure autoconf tool check to detect
if a shell outside mozilla-build will be picked up before our mozilla-build
shell is. configure fails if it detects a foreign copy of sh.

Comment 5

[Māris Fogels [:mars] (please needinfo)]

Attachment: Bug 1448349 - Add mozilla-build to autoconf $PATH on win32 r=#build

Sometimes users modify $PATH on Windows to install other MinGW-packaged
programs, such as Git. If the /bin directory for these other tools comes first
in the MozillaBuild shell $PATH then our Autoconf scripts will pick up the
Unix tools from the foreign MingGW package instead of the tools we package in
/mozilla-build. The foreign tools break and crash mach configure.

This patch ensures that the /mozilla-build directory always comes first in
$PATH when calling Autoconf during mach configure.

Comment 6

[Māris Fogels [:mars] (please needinfo)]

Attachment: Bug 1448349 - Warn about autoconf $PATH problems on win32 r=#build

Detect if Unix utilities on win32 are being picked up from a foreign
installation of MinGW, such as the tools packaged with Git for Windows.
If autoconf dies during ./mach configure and foreign tools are found
in $PATH then warn the user that their $PATH may need to change to fix
the problem.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7dcc2fa474a8
Warn about autoconf $PATH problems on win32 r=firefox-build-system-reviewers,chmanchester

Comment 8

[Andrei Ciure[:aciure]]

Backed out changeset 7dcc2fa474a8 (bug 1448349) for causing win2012 bustages

push that caused the backout: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception%2Cusercancel%2Crunnable&revision=7dcc2fa474a86bc2064b8ffa13e35f3ee7259344

backout: https://hg.mozilla.org/integration/autoland/rev/0f73b8cd0915bab32ed09d7c4c79477141e71469

Comment 9

[Mike Hommey [:glandium]]

My guess is the failure was due to the @imports('os'), which changed the meaning of many of the os.foo things in the function. Try @imports(_from='os', _import='environ')

Comment 10

[Māris Fogels [:mars] (please needinfo)]

My tests were passing locally and in Try, however :tomprince pointed out that I was running artifact builds in both. I was able to reproduce the failure using mach try --no-artifact.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/55d7f113d4ae
Warn about autoconf $PATH problems on win32 r=firefox-build-system-reviewers,chmanchester

Comment 12

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/55d7f113d4ae