767434 - mach: terminal control characters on Windows

Status: NEW

(Firefox Build System :: Mach Core, enhancement)

Description

[Gregory Szorc [:gps]]

Mach performs terminal magic and does things like add color output and add a footer displaying build progress.

Unfortunately, this doesn't work on Windows.

The reason it doesn't work on Windows is because mach is using Erik Rose's blessings Python package (http://pypi.python.org/pypi/blessings/) to perform the terminal interaction. This is effectively a wrapper layer on top of Python's curses module. And, the curses package is not distributed on Windows.

As :BYK pointed out in bug 751795 comment 20, http://www.lfd.uci.edu/~gohlke/pythonlibs/#curses exists. It is a port of Python's curses package to Windows. If this works, great! We should be able to drop it in mozilla-build and we are set!

There is a potential that this port is optimized for the Windows Command Prompt and won't work with the Cygwin/msys terminal emulator we use in mozilla-build. If that's the case, this bug will involve a little more work. We may have to write our own terminal interaction middleware. That could get ugly real fast. Time will tell.

Comment 1

[Gregory Szorc [:gps]]

In bug 505582, I've submitted a patch to switch the default mozilla-build terminal to mintty. That terminal is compatible with xterm. More info at https://code.google.com/p/mintty/wiki/CtrlSeqs.

If that patch gets accepted, we might be able to target xterm exclusively. Although, I'm not sure what terminals/terminal emulators are common in practice these days (I know I use xterm almost exclusively).

Comment 2

[Burak Yiğit Kaya [:BYK]]

Not actively working on it yet so moving self to CC.

Comment 3

[Mark Hammond [:markh] [:mhammond]]

I had a bit of a look at this and thought I'd share what I learned:

* curses for windows simply doesn't work in a native Python, due to it's attempt to use fcntl.ioctl to fetch the terminal width and height.

* mintty appears to be a native GUI program rather than a "console" program, so normal techniques for querying the size of the console don't work.

* The msys version of mintty seems to use the msys ioctl function to report screen size changes, and bash picks them up when only mintty is running.  Eg, when mintty is idle, you can resize the mintty window and COLUMNS is updated immediately.  However, if a Python program is running under mintty and you resize the window, COLUMNS never gets updated, even after the Python process terminates.  Once you again resize the window after Python terminates, COLUMNS reflects reality.

* As the Python used by mozbuild is the native win32 version rather than the msys one, I can't really see a way to have a child Python detect terminal size changes under mintty.  Maybe using ctypes to load the msys DLL with ioctl would work, but I didn't get that far (and that seems very fragile).

In short, I'm struggling to see how this could work for mintty (which I prefer over the other options).  conemu does use a windows console, so querying the screen size does work.  I haven't got as far as patching curses to query the windows screen size and seeing what blessings thinks of that - maybe later...

Comment 4

[Brian O'Keefe [:bokeefe]]

Attachment: Colors on windows

I was feeling ambitious, and thought I'd share this. I wanted to get colors somewhat useable on windows, so I started putting things in until it was functional.

This manages to get colors and formatting to "work", and enough positioning stuff that the footer looks okay. Granted, it "works" provided you have something that can deal with ANSI escape sequences [1], and your terminal is at least 124 characters wide (I haven't bothered to find the minimum width - that was just "good enough"). I've elected to just return 80x25 as the console size, instead of trying to figure it out intelligently.

This is probably not the right way to go about supporting color on Windows, but it works in the interim.

---

[1] I'm using AnsiCon (https://github.com/adoxa/ansicon), which handles most of the ANSI escape codes nicely. I have it installed with 'ansicon -i' so it "just works", but editing it into the Mozilla Build .bat files should work just as well.

Comment 5

[Steffen Wilberg]

Note that the mercurial color extension manages to colorize the output of e.g. hg qdiff.

The doc is here:
http://mercurial.selenic.com/wiki/ColorExtension

And the code is here:
http://selenic.com/repo/hg-stable/file/tip/hgext/color.py
It thought that could be helpful here.

Note that the auto-detection does not work for me, so you need to add 
  [color]
  mode = win32

as well as
  [extensions]
  color =

to your Mercurial.ini.