1353640 - Improvements to the build system code handling preprocessing

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Mike Hommey [:glandium]]

For bug 1349022, I did try builds getting the entire (well, most) code tree in preprocessed form. Essentially, I added %.$(OBJ_SUFFIX): %.i dependencies, and removed the condition hiding the .i rules.

This unveiled a couple issues:
- The .i rules don't work for generated sources.
- We uselessly create .deps files.

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1353640 - Factor the preprocessing rules to avoid all the redundancy.

Review commit: https://reviewboard.mozilla.org/r/126718/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/126718/

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1353640 - Avoid the preprocessor rules creating dependency files.

Review commit: https://reviewboard.mozilla.org/r/126720/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/126720/

Comment 3

[Mike Hommey [:glandium]]

Attachment: Bug 1353640 - Use more targetted vpath instead of broad VPATH for preprocessor rules.

VPATH applies to everything, but vpath applies to files matching the
pattern it's given.

Review commit: https://reviewboard.mozilla.org/r/126722/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/126722/

Comment 4

[Mike Hommey [:glandium]]

Attachment: Bug 1353640 - Also check sources in the objdir to preprocess generated sources.

Review commit: https://reviewboard.mozilla.org/r/126724/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/126724/

Comment 5

[Michael Shal [:mshal]]

Comment on attachment 8854744 [details]
Bug 1353640 - Factor the preprocessing rules to avoid all the redundancy.

https://reviewboard.mozilla.org/r/126718/#review129616

Comment 6

[Michael Shal [:mshal]]

Comment on attachment 8854745 [details]
Bug 1353640 - Avoid the preprocessor rules creating dependency files.

https://reviewboard.mozilla.org/r/126720/#review129624

::: config/rules.mk:1033
(Diff revision 1)
>  define PREPROCESS_RULES
>  _PREPROCESSED_$1_FILES := $$(call _group_srcs,$1,$2)
>  # Make preprocessed files PHONY so they are always executed, since they are
>  # manual targets and we don't necessarily write to $@.
>  .PHONY: $$(_PREPROCESSED_$1_FILES)
> -$$(_PREPROCESSED_$1_FILES): %.i: %.$1 $$(call mkdir_deps,$$(MDDEPDIR))
> +$$(_PREPROCESSED_$1_FILES): _DEPEND_CFLAGS=

The eval/call logic always has be slightly confused, but I don't think this is doing what you want. Since you are using $(COMPILE_CXXFLAGS) in the $(call) function, the cxxflags are expanded during the call and not during the rule processing. If you change $(eval) to $(info) you can see that the -MD and such flags are already in the rule. I think you want to use $$(COMPILE_CXXFLAGS) and the like below in order for this to take effect.. Otherwise I get this:

$ rm -rf .deps
$ make Unified_cpp_xpcom_ds1.i
/usr/bin/ccache /usr/bin/g++ ...
<built-in>: fatal error: opening dependency file .deps/.pp: No such file or directory
compilation terminated.
/home/mshal/mozilla-central-git/config/rules.mk:1066: recipe for target 'Unified_cpp_xpcom_ds1.i' failed
make: *** [Unified_cpp_xpcom_ds1.i] Error 1

Comment 7

[Michael Shal [:mshal]]

Comment on attachment 8854746 [details]
Bug 1353640 - Use more targetted vpath instead of broad VPATH for preprocessor rules.

https://reviewboard.mozilla.org/r/126722/#review129626

Comment 8

[Michael Shal [:mshal]]

Comment on attachment 8854747 [details]
Bug 1353640 - Also check sources in the objdir to preprocess generated sources.

https://reviewboard.mozilla.org/r/126724/#review129630

Comment 9

[Mike Hommey [:glandium]]

Comment on attachment 8854744 [details]
Bug 1353640 - Factor the preprocessing rules to avoid all the redundancy.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/126718/diff/1-2/

Comment 10

[Mike Hommey [:glandium]]

Comment on attachment 8854745 [details]
Bug 1353640 - Avoid the preprocessor rules creating dependency files.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/126720/diff/1-2/

Comment 11

[Mike Hommey [:glandium]]

Comment on attachment 8854746 [details]
Bug 1353640 - Use more targetted vpath instead of broad VPATH for preprocessor rules.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/126722/diff/1-2/

Comment 12

[Mike Hommey [:glandium]]

Comment on attachment 8854747 [details]
Bug 1353640 - Also check sources in the objdir to preprocess generated sources.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/126724/diff/1-2/

Comment 13

[Michael Shal [:mshal]]

Comment on attachment 8854744 [details]
Bug 1353640 - Factor the preprocessing rules to avoid all the redundancy.

https://reviewboard.mozilla.org/r/126718/#review129752

Comment 14

[Michael Shal [:mshal]]

Comment on attachment 8854745 [details]
Bug 1353640 - Avoid the preprocessor rules creating dependency files.

https://reviewboard.mozilla.org/r/126720/#review129754

Comment 15

[Michael Shal [:mshal]]

Comment on attachment 8854746 [details]
Bug 1353640 - Use more targetted vpath instead of broad VPATH for preprocessor rules.

https://reviewboard.mozilla.org/r/126722/#review129756

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f83bc4b3dfa6
Factor the preprocessing rules to avoid all the redundancy. r=mshal
https://hg.mozilla.org/integration/autoland/rev/388684b808db
Avoid the preprocessor rules creating dependency files. r=mshal
https://hg.mozilla.org/integration/autoland/rev/a614823d73f4
Use more targetted vpath instead of broad VPATH for preprocessor rules. r=mshal
https://hg.mozilla.org/integration/autoland/rev/7632e3ba8262
Also check sources in the objdir to preprocess generated sources. r=mshal

Comment 17

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/f83bc4b3dfa6
https://hg.mozilla.org/mozilla-central/rev/388684b808db
https://hg.mozilla.org/mozilla-central/rev/a614823d73f4
https://hg.mozilla.org/mozilla-central/rev/7632e3ba8262