777411 - "make -k check" should try harder to run more tests

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[(not currently active) Ted Mielczarek]

Currently if you run "make -k check" like our tinderboxes do, it will stop at the first test failure, which sucks. There are two reasons for this:
1) The CPP_UNIT_TESTS check:: rule exits on error in the loop:
http://mxr.mozilla.org/mozilla-central/source/config/rules.mk#108

Removing that line should fix that.

2) The way LOOP_OVER_DIRS is implemented means that as soon as a submake returns a failure result, execution will stop there (because make won't continue to execute more commands in a recipe when one has failed, even with -k):
http://mxr.mozilla.org/mozilla-central/source/config/rules.mk#351

The latter is a bit harder to fix. In a simple test I ran, if we conditionally (on k being present in $(MAKEFLAGS)), prepend - to the submake rule, like:
+-$(MAKE) $(if $(2),-C $(2)) $(1)

then make can continue on even if the submake fails. The downside here is that that will probably change the exit status of the overall make process, which might cause us to stop making the tree orange. Maybe we can hack around that.

Comment 2

[(not currently active) Ted Mielczarek]

I don't care enough about this anymore. We're just going to move everything out of make check.