818366 - Ability to pass through file descriptors to child processes

Status: RESOLVED FIXED

(Firefox Build System :: Mach Core, enhancement)

Description

[Gregory Szorc [:gps]]

Attachment: Execution mode to pass through file descriptors, v1

Mach isn't playing well with child processes that need full control over the standard file descriptors (stdin, stdout, stderr). This patch adds a process execution mode that uses subprocess instead of mozprocess to work around mozprocess's special handling of the standard file descriptors.

In my ideal world, mozprocess would offer this backdoor and we would never have to call subprocess. That being said, I think this is the path of least resistance today. If mozprocess ever develops this ability, we should convert mach to use mozprocess for everything again.

Comment 1

[(not currently active) Ted Mielczarek]

We went through this same thing for the same reason in bug 746239, and jhammel wound up with the same solution.

Comment 2

[Gregory Szorc [:gps]]

https://hg.mozilla.org/mozilla-central/rev/2d17a07b3113

Comment 3

[Walter Chen[:ypwalter][:wachen]]

This is causing failure in Jenkins CI for firefox OS automation in TPE CI server.

The reason of failure:
Jenkins CI is not using shell, and nothing matches in the following logic. It would throw an exception of "Could not detect environment shell!".

if 'SHELL' in os.environ:
    _current_shell = os.environ['SHELL']
elif 'MOZILLABUILD' in os.environ:
    _current_shell = os.environ['MOZILLABUILD'] + '/msys/bin/sh.exe'
elif 'COMSPEC' in os.environ:
    _current_shell = os.environ['COMSPEC']
else:
    raise Exception('Could not detect environment shell!')

The solution:
Do "export SHELL=/bin/bash", or just use injection of environment variable
http://release1-qa.corp.tpe1.mozilla.com:8080/job/B2G.master.unagi/7/

Comment 4

[(not currently active) Ted Mielczarek]

Walter: that doesn't actually seem relevant to this bug. It sounds like you're talking about this Pymake code:
http://mxr.mozilla.org/mozilla-central/source/build/pymake/pymake/util.py#38

The only supported build environment on Windows is the Mozillabuild shell. If you're trying to build in some other environment you are going to have a bad time.

Comment 5

[Jeff Hammel]

So, while we (=A*Team, Mozbase firing squad) have been fairly slow at addressing the many needed mozprocess bugs (see bug 881421 for a rough attempt at triage) I would like to aim (as priority/time permits) to make this sort of thing possible sans consumer refactor. Thoughts?