1385344 - dump_syms and 64-bit binaries

Status: RESOLVED WONTFIX

(Tecken :: General, task)

Description

[Peter Bengtsson [:peterbe]]

See https://bugzilla.mozilla.org/show_bug.cgi?id=1383067#c1

Basically, we need to do more when unpacking symbol files we download from Microsoft.

Comment 1

[Peter Bengtsson [:peterbe]]

Ted,
Following up on irc conversation...

* The dump_syms tecken uses [0] is different from your symsrv-fetch tool. dump_syms works on without being on Windows. 
* When I run a lookup of ntdll.pdb/D2D9EED62A1455D49CD6BC1D718463691/ntdll.sym it downloads https://msdl.microsoft.com/download/symbols/ntdll.pdb/D2D9EED62A1455D49CD6BC1D718463691/ntdll.pd_ and happily cabextracts that and dump_syms creates a .sym file that looks like this: [1]
* That output is different from https://s3-us-west-2.amazonaws.com/org.mozilla.crash-stats.symbols-public/v1/ntdll.pdb/D2D9EED62A1455D49CD6BC1D718463691/ntdll.sym
** The one on org.mozilla.crash-stats.symbols-public is 103KB and the one tecken makes is 59KB. 
** The one on org.mozilla.crash-stats.symbols-public's second line is `INFO CODE_ID A329D3A81DB000 ntdll.dll`

On IRC you said "no, it needs this code to do the right thing, unfortunately" meaning dump_syms code tecken uses needs to be done differently. This is a note to remind you to help me help you. 


[0] https://github.com/mozilla-services/tecken/blob/7c2f74494f8729cc72e74fa5bd47f755bda94f4b/docker/build_dump_syms.sh#L10
[1] https://gist.github.com/peterbe/04fb03dd9e3a3afb31d3bbb7b35bad15

Comment 2

[Peter Bengtsson [:peterbe]]

Also, you said to remind to file a bug regarding this; https://chromium.googlesource.com/breakpad/breakpad/+/master/src/common/windows/pdb_source_line_writer.cc#700

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm clearing the needsinfo on Ted. I don't know who's working on Symbols in this way anymore, but maybe Gabriele or Nathan knows?

Comment 4

[Gabriele Svelto [:gsvelto] (PTO)]

IIUC there's two problems here: the first one is that Tecken needs to fetch the EXE/DLL files and not just the PDB files. It's not clear to me if that's been fixed already or not. The second issue is that Tecken is using a version of dump_syms made by Ted because the dump_syms we have in the codebase relies on some Microsoft libraries and thus is not suitable for deployment in a Linux environment. This is a component that I'm rewriting in Rust (hopefully this month) so if I get that done we could replace both the version we have in mozilla-central and the one used by Tecken.

Comment 5

[Peter Bengtsson [:peterbe]]

Thanks for the update gsvelto!

Yeah, the "naive" dump_syms that Tecken has (compiled in Tecken's Dockerfile but not enabled to be used by default) assumes it can figure it out based on just the .sym file.

Internally, from that .sym file it knows what cab file to download from MS, and that gets turned into a .pdb file and lastly it executes dump_syms on it. However, there's no EXE/DLL involved. Just from 1 .sym symbol name (the missing one) to create another .sym file (uploaded if all works).

It all starts here: https://github.com/mozilla-services/tecken/blob/bc17c36e10a39c62144eb15e1d7ea0abb9f62530/tecken/download/views.py#L187-L191 but it's a bit hairy because of various "tricks" to use caching to prevent this from happening too much etc.

(n.b. clearning nfroyd's needinfo because I believe gsvelto answered it)

Comment 6

[Peter Bengtsson [:peterbe]]

What I never really understood was the code_file and code_id. These are query string parameters, from Socorro's symbolicator.
For example, I just noticed, in a snapshot of looking at https://symbols.mozilla.org/downloads/missing that there was a symbol download attempt for

atidxx64.pdb/49A2413755C14562B3075B7EC91A17EF1/atidxx64.sym?code_file=atidxx64.dll&code_id=5050F2036c3000

And I never understood how that code_file comes into play. Ted was planning to explain it to me when he upgraded dump_syms to work properly with 64-bit builds. But because I was eager I built something in Tecken and I'm pretty confident it doesn't do anything with the code_file and code_id around the use of dump_syms.

In other words, I don't think that they day gsvelto comes up with a replacement for dump_syms that Tecken is ready. I think it needs to do some more things to feed gsvelto's binary when it's ready.

Sorry if this my notes and comments are a bit disorganized. Better to write down some that to let it fade into no notes.

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

Bumping this over to the Tecken product where it belongs.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

I just wrote up bug #1641212 to remove this feature rather than finish it off. Marking this as WONTFIX.