1272278 - dump_syms failing to produce function names for Rust functions

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, defect)

Description

[(not currently active) Ted Mielczarek]

In bug 1268617, rillian made us start passing `-g` to rustc to get debug symbols. Unfortunately, he hit a problem in CI because dump_syms (which reads native debug symbols and outputs Breakpad text-format symbols) was missing function names for functions from Rust code on Windows. bug 1270091 fixed the thing that was breaking due to empty function names, but this is still a problem that we  need to fix.

bug 1268617 comment 6 has some brief analysis.

Breakpad uses Microsoft's DIA SDK to read information out of PDB files:
https://msdn.microsoft.com/en-us/library/x93ctkx8.aspx

Odds are the problem lies within `PDBSourceLineWriter::GetSymbolFunctionName`:
https://dxr.mozilla.org/mozilla-central/rev/3461f3cae78495f100a0f7d3d2e0b89292d3ec02/toolkit/crashreporter/google-breakpad/src/common/windows/pdb_source_line_writer.cc#957

https://dxr.mozilla.org/mozilla-central/rev/3461f3cae78495f100a0f7d3d2e0b89292d3ec02/toolkit/crashreporter/google-breakpad/src/common/windows/pdb_source_line_writer.cc#957

Comment 1

[(not currently active) Ted Mielczarek]

I started stepping through `GetSymbolFunctionName` in a debugger and found that the `get_undecoratedNameEx` call fails (unsurprising), but then the `get_name` call succeeds, handing back an empty string. :-/

Comment 2

[(not currently active) Ted Mielczarek]

Digging in a little more, it turns out the functions we're getting empty names from are from libstd...

Comment 3

[(not currently active) Ted Mielczarek]

Upstream patch for dump_syms: https://codereview.chromium.org/1985643004
I also wrote a patch for the Breakpad processor: https://codereview.chromium.org/1981933002 .

If that lands we can get that into the next Socorro production push, so that we'd be able to handle the existing symbol files with missing function names. We could also reprocess the current set of broken reports and they should get useful signatures.

I'd still like to figure out exactly why this is happening, but this should make it stop breaking our crash reporting.

Comment 4

[(not currently active) Ted Mielczarek]

dump_syms patch landed upstream: https://chromium.googlesource.com/breakpad/breakpad/+/67d5567177301d0c24303f26ad119ab7bd7fab40

I'll cherry-pick that into inbound.

Comment 5

[(not currently active) Ted Mielczarek]

https://hg.mozilla.org/integration/mozilla-inbound/rev/8be3eb37856c2609098b07d3f88ba482892c21b6
bug 1272278 - Don't let PDBSourceLineWriter::GetSymbolFunctionName return empty function names. r=mento

Comment 6

[(not currently active) Ted Mielczarek]

I punted "figure out a root cause" to bug 1273197.

Comment 7

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/8be3eb37856c