Support gcc/clang-like build path prefix map in generated code
Categories
(Firefox Build System :: General, enhancement)
Tracking
(firefox91 fixed)
| Tracking | Status | |
|---|---|---|
| firefox91 | --- | fixed |
People
(Reporter: nalexander, Assigned: nalexander)
References
(Blocks 2 open bugs)
Details
Attachments
(1 file, 3 obsolete files)
This ticket tracks substituting sentinels like "{topsrcdir}" and "{topobjdir}" into generated code in our build system.
The motivation is to achieve sccache hits across source and object directories by reducing "pointless" changes; it's part of the programme of using -fdebug-prefix-map, etc and similar flags to achieve reproducible builds. However this isn't really necessary for reproducible builds, since those tend to assume a fixed toolchain and filesystem, but it doesn't hurt.
In particular, we embed full paths into preprocessed sources and into generated XPIDL sources. We can stop doing that pretty cheaply!
|
Assignee | |
Comment 1•5 years ago
|
||
|
|
Assignee | |
Comment 2•5 years ago
|
||
The new option can be set in mozconfig files. It defaults to mapping
the objdir to "/topobjdir/" (or, on Windows, "c:\topobjdir", with the
same drive letter as the original objdir) and the srcdir to
"/topsrcdir/" (or "c:\topsrcdir", again with the same drive letter).
This should improve (s)ccache hits between object directories.
Depends on D18382
|
|
Assignee | |
Comment 3•5 years ago
|
||
ted, dmajor: this is what I was talking about in #build today. I'm not sure it's relevant, but maybe it is?
The next step in this process would be to pass -fdebug-prefix-map=... to relevant compilers and then to update .{gdb,lldb}init scripts to do the unmapping.
Neat! I think the specific case that I was investigating was fixed by merely the -EP switch, but the prefix map sounds like it could bring even more cache hits on top of that.
|
||
Comment 5•5 years ago
|
||
So part 1 looks totally reasonable. I'm less sure about part 2--where are we outputting file names in Preprocessor.py output currently? As dmajor mentioned sccache will use -EP when running the preprocessor internally, which leaves out the file markers that would normally cause preprocessor output to be different between different srcdirs.
|
||
Comment 6•4 years ago
|
||
Part 2 has been done by bug 1528892 and bug 1618775, see the code at:
|
|
Assignee | |
Updated•3 years ago
|
|
|
Assignee | |
Updated•3 years ago
|
|
|
Assignee | |
Comment 7•3 years ago
|
||
For "c" (i.e., gcc, clang, and clang-cl), this configures the
-f{debug,macro}-path-prefix flags. We'd prefer to use
-ffile-path-prefix, but it seems that clang-cl does not recognize
that flag.
For "rust" (i.e., rustc/cargo), this configures --remap-path-prefix.
This is one step toward getting sccache hits across source and
object directories.
|
||
Updated•3 years ago
|
|
|
Assignee | |
Comment 8•3 years ago
|
||
The big win with the patch I've just submitted is that __FILE__ macros no longer embed build environment-specific paths, opening the possibility for cross source directory sccache hits. Actually realizing those sccache hits can take some work, though; I'm hoping to push on this more in the near future.
|
|
Assignee | |
Comment 9•3 years ago
|
||
|
|
||
Updated•3 years ago
|
|
||
Comment 10•3 years ago
|
||
Pushed by nalexander@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/7780b7c82d22 Add --enable-path-remapping producing compiled objects with generic paths. r=glandium
|
||
Comment 11•3 years ago
|
||
| bugherder | ||
Description
•