Closed Bug 535071 Opened 10 years ago Closed 10 years ago

Linux crash report file ID doesn't match file ID from dumped symbols

Categories

(Toolkit :: Crash Reporting, defect)

x86
Linux
defect
Not set

Tracking

()

RESOLVED FIXED

People

(Reporter: dholbert, Assigned: ted)

References

()

Details

(Keywords: regression)

My crash reports from today's nightlies haven't had useful stack information -- instead of function names, they just have module name + offset.

For example: I just triggered the crash from bug 535004, using a nightly from yesterday and a nightly from today.  Here are the results:

Yesterday's nightly (20091214030830):
http://crash-stats.mozilla.com/report/index/801b2e3e-e575-4240-8c07-716822091215
* Title has signature "[@ nsSMILCSSProperty::GetBaseValue() const ]"
* Stack has symbol information (names of functions)

Today's nightly (20091215030948):
http://crash-stats.mozilla.com/report/index/94e9fb23-5eb6-416d-a58e-729162091215
* Title has signature  "[@ libxul.so@0x7fa124 ]"
* Stack has no symbol information -- just has e.g. "libxul.so@0x7f9c3e" at each stack level.
Pushlog for regression range:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=44c392db6672&tochange=96e8d529b2d3

Just got a new crash report from each build again, with the same results.

Yesterday's nightly (with symbols):
http://crash-stats.mozilla.com/report/index/960fd911-0d3b-4cdd-aad4-6706f2091215
Today's nightly (no symbols):
http://crash-stats.mozilla.com/report/index/0a8bee82-04b9-4cb5-957c-9d9422091215

The changes for bug 514188 ("sync to breakpad revision 437 to pick up Linux client rewrite") sounds particularly suspicious...  I guess the out-of-process-plugins patch-bomb seems suspicious too, if only because of its size.  I'm blaming bug 514188 for now, though...
Component: General → Breakpad Integration
Product: Core → Toolkit
QA Contact: general → breakpad.integration
There's something...odd about that second crash report. We have symbols for the build with that build ID:
http://symbols.mozilla.org/firefox/firefox-3.7a1pre-Linux-20091215030948-symbols.txt

However, the libxul.so there has the ID:
libxul.so/6429391C3C6500D5A2B4C500E40E1A000/libxul.so.sym

but the libxul.so from your crash report has the ID:
468C329B3C6500D5A2B4C500E40E1A000

Those don't match. So either you're running a mixed-up build, or something is broken between the symbol dumping and crashing bits.
Jim: don't suppose any of your symbol dumper changes could have caused something weird to happen here?
Ugh. The Linux client rewrite definitely broke this. Apparently the Linux File ID code changed to just calculate a hash of the first page of the file, which changes if you strip the file or anything. I guess I'll back out until I can figure out how to fix it.
I filed an upstream issue on this brokenness:
http://code.google.com/p/google-breakpad/issues/detail?id=357
Backed out, bug 514188 reopened. Apparently bsmedberg is going to get some new nightlies spun today to fix some e10s stuff, so the new nightly today should be better.

I'll try to get this sorted out upstream so we can re-land, since the new code is otherwise much better.
Assignee: nobody → ted.mielczarek
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Summary: Crash reports don't include useful stack information, starting with 20091215 nightly → Linux crash report file ID doesn't match file ID from dumped symbols
Was it expected that your backout would actually *cause* this, for Linux talos crashes? Prior to your backout, and for one of the two that tbpl claims were from your backout, the bug 528708 crashes all had the libgtk-x11-2.0.so.0.1200.0 signature, then for one from the backout and all following ones, they're (assuming they really are the same thing) all just libxul.so + 0xddf84 (or 0xddf44, or thereabouts).
Uh, no. The backout should have just reverted us to the previously existing Breakpad code. (I forgot to look into what happened with the debug symbols on tinderbox, I would expect things to continue working.)
You need to log in before you can comment on or make changes to this bug.