795775 - mach fails to build using mintty on Windows

Status: RESOLVED DUPLICATE of bug 796875

(Firefox Build System :: Mach Core, enhancement)

Description

[Mark Hammond [:markh] [:mhammond] [PTO until Sept 7]]

I'm trying to use mach to build under the "mintty" terminal on Windows.  This works fine using pymake explicitly, but using mach hangs for a minute or so after printing the first line, then displayed the following errors:

$ ./mach -v build
0.22 C:/mozilla-build/msys/bin/sh.exe -c o:/src/mm/mozilla-hg/mozilla-central/build/pymake/make.py -f client.mk -j8 -s
No handlers could be found for logger "pymake.data"
Exception in thread Thread-4:
Traceback (most recent call last):
  File "c:\mozilla-build\python\lib\threading.py", line 552, in __bootstrap_inner
    self.run()
  File "c:\mozilla-build\python\lib\threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "o:\src\mm\mozilla-hg\mozilla-central\testing/mozbase/mozprocess\mozprocess\processhandler.py", line 683, in _processOutput
    self.processOutputLine(line.rstrip())
  File "o:\src\mm\mozilla-hg\mozilla-central\testing/mozbase/mozprocess\mozprocess\processhandler.py", line 648, in processOutputLine
    handler(line)
  File "o:\src\mm\mozilla-hg\mozilla-central\python/mozbuild\mozbuild\base.py", line 310, in handleLine
    line = line.decode(sys.stdout.encoding)
TypeError: decode() argument 1 must be string, not None

Traceback (most recent call last):
  File "./mach", line 48, in <module>
IO Completion Port unexpectedly closed
mach.run(sys.argv[1:])
  File "o:\src\mm\mozilla-hg\mozilla-central\python/mach\mach\main.py", line 133, in run
    fn(**stripped)
  File "o:\src\mm\mozilla-hg\mozilla-central\python/mach\mach\build.py", line 45, in build
    log=False, print_directory=False)
  File "o:\src\mm\mozilla-hg\mozilla-central\python/mozbuild\mozbuild\base.py", line 256, in _run_make
    fn(**params)
  File "o:\src\mm\mozilla-hg\mozilla-central\python/mozbuild\mozbuild\base.py", line 279, in _run_command_in_srcdir
    self._run_command(cwd=self.topsrcdir, **args)
  File "o:\src\mm\mozilla-hg\mozilla-central\python/mozbuild\mozbuild\base.py", line 333, in _run_command
    status = p.wait()
  File "o:\src\mm\mozilla-hg\mozilla-central\testing/mozbase/mozprocess\mozprocess\processhandler.py", line 722, in wait
    return self.proc.wait()
  File "o:\src\mm\mozilla-hg\mozilla-central\testing/mozbase/mozprocess\mozprocess\processhandler.py", line 134, in wait
    self.returncode = self._wait()
  File "o:\src\mm\mozilla-hg\mozilla-central\testing/mozbase/mozprocess\mozprocess\processhandler.py", line 421, in _wait
    raise OSError(err)
OSError: IO Completion Port failed to signal process shutdown

Comment 1

[Gregory Szorc [:gps]]

This is 2 bugs:

1) Not properly detecting when sys.stdout.encoding == None
2) mintty bustage

mintty used to work. I wonder if it broke when I switched mach to build through client.mk. hmmm...

Comment 2

[Gregory Szorc [:gps]]

Well, the lack of output isn't a logging problem (like build output not being captured by a log handler).

Instead, it appears this is a fundamental problem with interaction with mintty. I hacked up mozbuild's _run_command helper and the mozprocess processOutputLine handler is not being invoked. Why, I dunno. I also noticed that if a Ctrl-c during a build I don't see a stack trace. But if I raise an exception in _run_command I do see a traceback. It definitely sounds like mozprocess and mintty aren't playing well together...

Comment 3

[Mark Hammond [:markh] [:mhammond] [PTO until Sept 7]]

hrm - while trying to debug some other strange issue with mach, I ended up doing:

% ./mach -v build > out 2>&1

from normal bash (ie, no mintty involved) and ended up with the same error.  Without the redirection I don't see this problem (just the other problem I'm trying to track down :) So it might not be mintty specific at all...

Comment 4

[Mark Hammond [:markh] [:mhammond] [PTO until Sept 7]]

This was temporarily resolved by the patch in bug 795670, but in general is a dupe of 796875.  A build using mach now completes - there might be other issues (particularly around output buffering etc), but the "fails to build" is no longer true, so closing.