492903 - Programmatically generate more in-depth OS libraries list for symbols from our binaries

Status: RESOLVED FIXED

(Camino Graveyard :: General, defect)

Description

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

In bug 492238 comment 3, Stuart writes:

> Sadly, it doesn't end there. First, we slipped a few libraries in with
> OTHER_LDFLAGS in Camino: libm and libpthread. The much nastier part though is
> that starting from this list, a thorough list would be the transitive closure
> of all frameworks and libraries that are linked starting from the list you
> have. You can see what a library loads with otool -L; to pick one at random
[...]
> Obviously doing this by hand would be soul-crushing, so lets do this: go ahead
> with the list you have, but file a bug (and assign it to me, I suppose) to make
> a script that will start with a list of binaries and generate the complete list
> programatically. I'll hopefully get to it at some later date; strangely
> (sadly?), that will not be the first time I've written a script to do this...

Some follow-on to that:

1) libm and libpthread are links to libSystem, which in turn is a link to libSystem.B, so they're actually covered in the current system.

2) We can't *solely* use the programmatically-generated list of libraries/frameworks to dump, because 
  a) There are a reasonable number of things that get loaded (and even have 
     crashed) that aren't called/linked directly by Camino, e.g. plug-ins and 
     input methods.  These are not necessarily always interesting crashes, but 
     if we're crashing in them, symbols are always better than no symbols.
  b) While I didn't go out of my way to make sure I included every framework 
     that other products or branches might use, I also didn't just ignore 
     things I knew were used but which didn't also come up in my Camino-focused 
     queries (e.g. Tb's Spotlight importer, CoreText on 1.9.x>0).
For these cases we'd still need to be able to supplement the programmatic determination with some hard-coded files.

Comment 1

[Stuart Morgan]

(In reply to comment #0)
> For these cases we'd still need to be able to supplement the programmatic
> determination with some hard-coded files.

Yep, I'll make it start with a seed list of whatever we want, and then generate the whole tree downward from those.

Comment 2

[Stuart Morgan]

Attachment: split library lists out of the symbol generation script

This is prep for the real patch; all it does is split the list of libraries out into separate files, and read them from there (so now it would be run as 'generate_macosx_symbols ... < libraries_10_<whatever>.txt')

By itself this looks worse, but with the next part the list will be autogenerated, so the duplication won't be a problem. I'm doing in two parts so that we'll get a usable diff between Smokey's list and the auto-generated one to help catch any mistakes.

Comment 3

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Comment on attachment 382043 [details] [diff] [review]
split library lists out of the symbol generation script

r=ardissone

Comment 4

[Stuart Morgan]

Attachment: dependency generator

This is the dependency generator script, the extra seed files, and the result of me running it on a bunch of moz apps to make a new 10.5 library file.

There are a few things that deliberately aren't in my list that are in Smokey's:
- ExceptionHandling (turns up in mxr, but is actually only linked for debug builds).
- Various BridgeSupport stuff, since we aren't aware of actually needing it as I understand it.
- Quartz.framework, which apparently nothing we use actually links to (just QuartzCore).

(It looks like a lot more diffs because the otool output includes the /Versions/<foo>/ part of the path).

Comment 5

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Comment on attachment 382047 [details] [diff] [review]
dependency generator

r=ardissone with the new RealPlayer path per irc.

Comment 6

[Stuart Morgan]

All landed on CVS trunk.