451057 - make -k doesn't attempt every file in the subtree

Status: RESOLVED INVALID

(Firefox Build System :: General, defect)

Description

[Zack Weinberg (:zwol)]

The -k switch to make stands for "keep going" - it's supposed to build everything it possibly can, giving up only on files whose compilation causes errors and their direct transitive dependencies.  If you try to use this at the top level of a Mozilla object tree, it gives up after the first top-level directory that had an error somewhere inside.  This is wrong.  It should go on to the next one in the list.  After all, most of the top-level directories are pretty isolated from each other.

Comment 1

[Benjamin Smedberg]

Well, I doubt that is actually true. But how do you propose to fix it? Isn't this part of the standard $(MAKE) recursion mechanism?

Comment 2

[Zack Weinberg (:zwol)]

(In reply to comment #1)
> Well, I doubt that is actually true. 

Try it - introduce a compile error deep in content/ and do a from-scratch build with -k.  I'm not sure exactly how many directories got skipped but it looked like at least half of the overall tree.

> But how do you propose to fix it? Isn't  this part of the standard 
> $(MAKE) recursion mechanism?

No.  A standard recursive make construct like

all: $(subdirs)
$(subdirs):
        $(MAKE) -C $@ all

will attempt every directory in $(subdirs) even if one of the early ones has an error.  I think the problem is that config/rules.mk isn't doing that, but instead this:

default all alldep::
        $(EXIT_ON_ERROR) \
        $(foreach tier,$(TIERS),$(MAKE) tier_$(tier); ) true

Of course, it may not be simple to switch to a strictly dependency-based rule, even if that's the abstract right thing.  A band-aid fix would be to suppress $(EXIT_ON_ERROR) if -k is in effect.

Comment 3

[Benjamin Smedberg]

What I mean by "is not true" is that directories are largely independent. Certainly a malformed IDL file can cause large swaths of the tree to fail later on.

I certainly don't think we want to suppress the EXIT_ON_ERROR bit entirely, because then the toplevel make process wouldn't exit with the correct error code. But perhaps you could collect and propagate the failing error code in that case at the end of the loop.

But really, I'm not sure it's worth the effort. What information are you trying to get out of make -k for the entire tree?

Comment 4

[Zack Weinberg (:zwol)]

> What information are you trying to get out of make -k for the entire tree?

I've just done a hg pull underneath a MQ patch that I know to be incomplete, and I want to fire off a rebuild of as much as I possibly can given that some files are going to fail, while I wander away from the keyboard to get coffee.

Comment 5

[Zack Weinberg (:zwol)]

On further experimentation, this isn't just a problem at top level; a rebuild with -k from the top of (e.g.) layout does not attempt every file that it could have attempted.  Also, I have another, more personally important, use case: if make -k worked correctly, I could change an interface in a widely included header, such as nsLayoutUtils.h, and then do one recompile with -k and have the error messages point out every place that needs to be changed.  As is, I may have to cycle dozens of times, each cycle finding only one additional place to fix.

Comment 6

[Gregory Szorc [:gps]]

I don't think there's much we can do here.