967696 - mach does not stop early when python unit tests fail

Status: NEW

(Firefox Build System :: Mach Core, enhancement)

Description

[Nathan Froyd [:froydnj]]

Running:

mach build check

cheerily runs all the python unit tests, even if a test somewhere in the middle fails.  In contrast,

blah/blah/pymake.py -f Makefile check; or
make check

stops after the first failed unit test.  This behavior can make it difficult to determine what test has failed due to limited terminal scrollback.

Comment 1

[Gregory Szorc [:gps]]

make check's early termination behavior is what we want to avoid because it masks multiple failures and can result in multiple try pushes since you have to play whack-a-mole. What you really want is a summary of failures. Bug 917363 should hopefully facilitate that.