1473331 - node detection fails on Windows systems due to environment issues

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Nathan Froyd [:froydnj]]

As reported in bug 1471028, some Windows systems (or some Python versions on Windows?) fail configure while looking for node with:

 0:15.97 checking for nodejs... c:/mozilla-build/node-v8.11.1-win-x64/node.exe
 0:15.98 checking for node.js version... Traceback (most recent call last):
 0:15.98   File "z:/Mozilla/mozilla-central/configure.py", line 123, in <module>
 0:15.98     sys.exit(main(sys.argv))
 0:15.98   File "z:/Mozilla/mozilla-central/configure.py", line 29, in main
 0:15.98     sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 428, in run
 0:15.98     func(*args)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 474, in _value_for
 0:15.98     return self._value_for_depends(obj, need_help_dependency)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\util.py", line 944, in method_call
 0:15.98     cache[args] = self.func(instance, *args)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 483, in _value_for_depends
 0:15.98     return obj.result(need_help_dependency)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\util.py", line 944, in method_call
 0:15.98     cache[args] = self.func(instance, *args)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 123, in result
 0:15.98     return self._func(*resolved_args)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 1003, in wrapped
 0:15.98     return new_func(*args, **kwargs)
 0:15.98   File "z:/Mozilla/mozilla-central/build/moz.configure/util.configure", line 430, in wrapper
 0:15.98     return func(*args)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 1003, in wrapped
 0:15.98     return new_func(*args, **kwargs)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 735, in wrapper
 0:15.98     ret = template(*args, **kwargs)
 0:15.98   File "z:/Mozilla/mozilla-central/build/moz.configure/checks.configure", line 58, in wrapped
 0:15.98     ret = func(*args, **kwargs)
 0:15.98   File "z:\Mozilla\mozilla-central\python\mozbuild\mozbuild\configure\__init__.py", line 1003, in wrapped
 0:15.98     return new_func(*args, **kwargs)
 0:16.00   File "z:/Mozilla/mozilla-central/build/moz.configure/node.configure", line 29, in nodejs_version
 0:16.00     stderr=subprocess.STDOUT)
 0:16.00   File "c:\mozilla-build\python\Lib\subprocess.py", line 216, in check_output
 0:16.00     process = Popen(stdout=PIPE, *popenargs, **kwargs)
 0:16.00   File "c:\mozilla-build\python\Lib\subprocess.py", line 394, in __init__
 0:16.00     errread, errwrite)
 0:16.00   File "c:\mozilla-build\python\Lib\subprocess.py", line 644, in _execute_child
 0:16.00     startupinfo)
 0:16.00 TypeError: environment can only contain strings
 0:16.02 *** Fix above errors and then restart with\
 0:16.02                "c:/mozilla-build/bin/mozmake.EXE -f client.mk build"
 0:16.03 mozmake.EXE: *** [client.mk:127: configure] Error 1

Such a helpful error message.

Comment 1

[h-h]

I can confirm the exact same error message and traceback (verified)!

I tryed clobbering and it also didn't work.

Grr! Now, I have to repeat the hour-long build. :/

Comment 2

[Jorg K (CEST = GMT+2)]

For a "wrong" workaround see bug 1471028 comment #11.

Comment 3

[Nathan Froyd [:froydnj]]

I think the subprocess.check_output call here:

https://hg.mozilla.org/mozilla-central/rev/fe18414fe685#l1.32

should be:

  out = check_cmd_output(node, '--version', env=env)

since check_cmd_output performs some environment sanitization.  Can people seeing this try that instead?

Comment 4

[Richard Marti (:Paenglab)]

Changing

out = subprocess.check_output([node, '--version'], env=env, stderr=subprocess.STDOUT)

to

out = check_cmd_output(node, '--version', env=env)

let it build.

Comment 5

[h-h]

@ Nathan Froyd: It works:

> 0:24.47 checking for nodejs... c:/mozilla-build/node-v8.11.1-win-x64/node.exe
> 0:24.53 checking for node.js version... 8.11.1
> 0:24.64 checking for tar... c:/mozilla-build/msys/bin/tar.exe

---

BTW: Calls in the Mozilla build console:
> $ which python node
> /c/mozilla-build/python/python.exe
> /c/mozilla-build/node-v8.11.1-win-x64/node.exe
> $ python --version && node --version
> Python 2.7.15
> v8.11.1

Comment 6

[Nathan Froyd [:froydnj]]

To be clear, the check_cmd_output change was made without changing the assignment of:

  env[b'NODE_DISABLE_COLORS'] = b'1'

correct?

Comment 7

[Richard Marti (:Paenglab)]

Correct.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/b66f3991034c
fix environment issues when checking node version; r=me, a=bustage

Comment 9

[Nathan Froyd [:froydnj]]

Thanks for the remote debugging, all!