671615 - vpath lookups get reset

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Steve Fink [:sfink] [:s:]]

Attachment: Test case that works in gmake, fails in pymake

I'm attaching a zip file of a directory containing a Makefile that works with gmake but fails with pymake. This is probably not a problem for properly-written makefiles, but I think it should either work or (preferably, assuming it's possible) spew out a warning making it much easier to identify the problem in the makefile.

From spending far too much time digging into this, what appears to be happening is this:

I have a target 'foo.obj' that depends on $<. $< needs to be found via VPATH. That works fine, until I add another rule where the same string is a target. Then because that target gets remade, its .vpathtarget gets reset to .target (I hope I'm remembering the field names correctly.) This happens at just the wrong time, so that when the 'foo.obj' rule is evaluated, $< has forgotten its .vpathtarget path and furthermore, because .vpathtarget is set, it doesn't attempt to re-resolve. So it looks for $< in the wrong directory.

The specific problem I was having was that I had a rule

$(CPPSRCS:.cpp=.obj) : generated.h

(with no commands) to make all .obj files in a directory depend on a generated .h file. Unfortunately, not all of $(CPPSRCS) ended in .cpp; there were some that ended in .cc instead. As a result, I ended up with the equivalent of

foo.cc : generated.h

which is incorrect, and mysteriously caused a completely different rule to break (because it didn't have the VPATH-found path for $>).

So one option would be to fix the bug. What would be cool is if pymake could detect that something fishy was going on and give a warning. I haven't thought too hard about that.

With respect to the bug, it seems a little odd that $< is looked up as a target in the 'makefile' data structure. It's a prerequisite, not a target. But my brain is too small to actually understand what's going on there; everything I know is the result of a few hours of debugging and lots of cursing about the crappy state of Python debugging tools. (Or perhaps the crappy state of my knowledge about Python debugging tools.)

Comment 1

[Steve Fink [:sfink] [:s:]]

Oops. You'll need to explicitly try to make 'foo.obj'. It doesn't come first in my test makefile.

Comment 2

[Benjamin Smedberg]

The fundamental difference here is:

make -d foo.obj
...
   Finished prerequisites of target file `foo.cc'.
   Prerequisite `generatedHeader.h' is newer than target `foo.cc'.
  No commands for `foo.cc' and no prerequisites actually changed.
  No need to remake target `foo.cc'; using VPATH name `subdir/foo.cc'.
 Finished prerequisites of target file `foo.obj'.

make.py -d foo.obj
...
INFO:pymake.data: Searching for implicit rule to make 'foo.cc'
INFO:pymake.data: Couldn't find implicit rule to remake 'foo.cc'
INFO:pymake.data: Considering target 'generatedHeader.h'
INFO:pymake.data: Searching for implicit rule to make 'generatedHeader.h'
INFO:pymake.data: Couldn't find implicit rule to remake 'generatedHeader.h'
INFO:pymake.data:Remaking foo.obj using rule at /builds/testcases/makefiles/make-vpath/mk/Makefile:6:0: target doesn't exist or is a forced target

So pymake assumes that if you mention a file as a target (as you did in this makefile), then it is never found via the vpath. gmake assumes that if you mention a file as a target *with commands* then it is not found via the vpath.

Comment 3

[Benjamin Smedberg]

This is very weird. gmake requires us to not use the vpath in this case:

http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/6ba5ffef40fd/tests/vpath.mk#l11

This is very similar to your current case except that there is a rule to remake `test.objtest`. After some experimentation, I've determined that it is not the mere presence of the rule which resets the vpath: the vpath for `tfile3` is only reset if we actually remake `test.objtest`.

This highly unintuitive behavior is not something I'd like to replicate in pymake. I'm happy to accept a warning message, but I really have no clue how you'd determine the conditions here without catching other things.

Comment 4

[Benjamin Smedberg]

Attachment: Testcase in tree-format as a patch

Here's the testcase in a form that fits in the automated testsuite.

Comment 5

[Steve Fink [:sfink] [:s:]]

I think pymake is dead now.