Closed Bug 1042476 Opened 10 years ago Closed 7 years ago

'make buildsymbols' can skip symbols dump if they're already in the archive

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
x86
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1337986

People

(Reporter: nthomas, Unassigned)

References

Details

This is a sketch, the context is dep builds on the releng automation.

make buildsymbols seems to dump every binary, every time, which can take several minutes. A lot of binaries don't change every build, and we may have already dumped them on a previous job. Windows is the exception IIRC.

So pass symbolstore.py the dist/crashreporter-symbols/blaaaaah.symbols.txt that exists from last time. Find the binaries as usual, and get their magic id, then consult the txt file. For changed binaries, remove the existing files in the crashreporter-symbols dir, then dump the new symbols there. Remove any deprecated files.

Bonus points for keeping the blah-crashreporter-symbols-full.zip and blah-crashreporter-symbols.zip files around instead of deleting them at the start, then removing/adding files as necessary.
... or move part of the logic from symbolstore.py to a makefile...
The problem is that to get the magic ID right now you need to run dump_syms, so you don't really save much. It's possible to write a smaller utility to grab the ID, but it's a bit of a PITA to get. Maybe it's worth the cost?

FWIW, 90% of the time is spent dumping symbols from libxul, which is likely to have changed on every build. (We dump everything in parallel now, so not dumping other libraries isn't likely to make any difference.)
Also, humorously, bug 1033938 argues against keeping the zip files, as we're not currently rm'ing the symbols-full.zip and it gets corrupted sometimes.
I get your point about libxul/XUL changing a lot, particularly on tree's like m-i and m-c. On b2g-inbound where this is a lot of gaia changes where it may actually help. What prompted this bug was noticing a b2g gecko build on mac where the compile took 5 mins, and buildsymbols was nearly 7. WONTFIX ?
Ah, that's a good point. We used to build a file_id tool for Linux builds that just dumped out the ID of a binary:
http://mxr.mozilla.org/mozilla-esr17/source/toolkit/crashreporter/fileid/file_id.cc

We could revive that (and make it work on all platforms) and modify the symbolstore.py script to be smart about not re-dumping things it doesn't need to.
catlee was asking about this the other day. I think it's feasible, we would just need to get the ID out without running dump_syms, and then skip dumping for binaries whose ID hasn't changed. Now that I think about it, nowadays we should be able to get the ID for binaries without using special tools, it's all in the binary headers:

Linux:
$ readelf -n ../debug-mozilla-central/toolkit/library/libxul.so | grep "Build ID"
    Build ID: 91d9b0b504cd46224b3ce7069a271735992231b3

Mac:
$ otool -l ../debug-mozilla-central/toolkit/library/XUL | grep uuid
    uuid CE4343B9-D4F9-3B90-878F-F3EC6E168FA7

Win:
$ dumpbin -HEADERS ../debug-mozilla-central/toolkit/library/xul.dll | grep RSDS
    554CD6FD cv           4F 078CE440  78CD240    Format: RSDS, {C7D95017-4631-4BB2-9415-9462EEBAF9D
5}, 1, c:\build\debug-mozilla-central\toolkit\library\xul.pdb
catlee also asked about the feasibility of having a cache of dumped symbols outside of the objdir, so that we could get this benefit for non-dep builds. This could work for Linux, where the UUID used in the Build ID is a hash of the text section. For Windows and Mac those are actually GUID/UUIDs generated by the toolchain at build time, so there's no way to know if the symbols would match short of dumping them.
except libxul contains the buildid, so every build has a different libxul.
(In reply to Mike Hommey [:glandium] from comment #8)
> except libxul contains the buildid, so every build has a different libxul.

where, by buildid, I mean *our* buildid, the one in $objdir/config/buildid. Fortunately, that's not in the text section, so it doesn't affect the build id on linux, a "simple" hash of the binary is not going to cut it for windows and mac considering *our* buildid.
Are we able to remove 'make buildsymbols' as a separate step, and instead run it after a program or shared library is linked? Something like:

$(SHARED_LIBRARY): $(OBJS) ...
    link $@
    buildsymbols $@

make is already only going to re-link things when the inputs have changed, and at that point we know we need to re-run buildsymbols, right?
We did talk about this. The only downside there is that we add time to the build step where we could be doing it in parallel with other tasks like packaging. Maybe we could do a half-measure and just have buildsymbols keep a state file for each binary, like $objdir/_symbols/libxul.so.dumped, and only dump symbols if that's older than the binary?
libxul is one of the last things built in the "compile" tier and it takes the longest to dump symbols. If we dump symbols during "compile," that will just delay the start of the "libs" tier.

I like Ted's idea of a tracking file. Then we can move buildsymbols to occur concurrently with other tasks, like test archive generation. As part of that, we'll almost certainly need to create multiple make targets for each file so we don't spawn too many processes and disregard make's -j flag (symbol dumping does its own forking).
The more I think about this the more I wonder if we shouldn't just be working to figure out how to integrate more of our post-build steps into the normal build DAG so that we can do things like dump symbols immediately after linking without it then blocking things that run after the build. I guess some of this is breaking down the build tiers, and some of it might be figuring out how to shift work from standalone steps into part of the build. Take packaging, for example. We could be staging the package contents as part of the build pretty easily (and we've discussed making dist/bin be the staging area in the past). For `buildsymbols`, if we did the dumping and the PDB compression as post-link steps, the `buildsymbols` step itself would just be "put these files into a zip".
chmanchester fixed this in bug 1337986.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → DUPLICATE
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.