920781 - Mach should use the python that called it when calling other things

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Dave Townsend [:mossop]]

0:00.33 C:/mozilla-build/msys/bin/sh.exe -c c:/mozilla/source/trunk/build/pymake/make.py
    -j4 -s install-tests
     0:01.42 /usr/bin/sh: c:/Users/Dave: No such file or directory
     0:01.42 c:\mozilla\build\nightly\Makefile:68:0: command 'MOZBUILD_BACKEND_CHECKED= c:/Users/Dave Townsend/python/Scripts/python.exe c:/mozilla/source/trunk/build/pymake/pymake/../make.py -C js/src backend.RecursiveMakeBackend.built' failed, return code 127
     0:01.42 <install-tests>: Found error

Here mach has decided to use the python in the PATH to run pymake and fails because it can't handle spaces in the path. The python I used to run mach here was the one from mozilla-build and doesn't have spaces in the path. It should just use that one.

Comment 1

[Gregory Szorc [:gps]]

Yeah, this is a real bug. base.py's run_make() will invoke make.py as a command. It's shebang is "#!/usr/bin/env python", which is not ideal.

Comment 2

[Gregory Szorc [:gps]]

Attachment: Invoke pymake with virtualenv or calling python, not $PATH python

Will need to test this before landing.

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 810174 [details] [diff] [review]
Invoke pymake with virtualenv or calling python, not $PATH python

Please report how this works.

Comment 4

[Mike Hommey [:glandium]]

Comment on attachment 810174 [details] [diff] [review]
Invoke pymake with virtualenv or calling python, not $PATH python

Review of attachment 810174 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/base.py
@@ +464,5 @@
> +            ve_python = self.virtualenv_manager.python_path
> +            if os.path.exists(ve_python):
> +                return [ve_python, make_py]
> +            else:
> +                return [sys.executable, make_py]

That doesn't look like a great thing to do. Because if that runs before configure (in case of a clobber build), it's going to use system python for the entire build. Further builds will use the virtualenv one. The discrepancy is going to hurt.

Comment 5

[Gregory Szorc [:gps]]

So, the only way we can reliably use the virtualenv python is if the virtualenv python is guaranteed to exist. And that means moving virtualenv population out of configure and into client.mk or mach. That's something I've been wanting to do, but that's not something I'd like to block this bug.

Will you accept a patch that always uses sys.executable?

Comment 6

[Mike Hommey [:glandium]]

(In reply to Gregory Szorc [:gps] from comment #5)
> Will you accept a patch that always uses sys.executable?

yes

Comment 7

[Gregory Szorc [:gps]]

Attachment: Invoke pymake with virtualenv or calling python, not $PATH python

I removed the virtualenv from the equation.

Comment 8

[Mike Hommey [:glandium]]

Comment on attachment 810291 [details] [diff] [review]
Invoke pymake with virtualenv or calling python, not $PATH python

Review of attachment 810291 [details] [diff] [review]:
-----------------------------------------------------------------

Don't forget to adjust the commit message.

Comment 9

[Gregory Szorc [:gps]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/5897df7c56e2

Comment 10

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/5897df7c56e2