1768863 - add support for code id and code file to symbolication api

Status: RESOLVED WONTFIX

(Eliot :: Symbolication, enhancement, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

For missing Microsoft symbols, we need the code id and code file to find them. Because of that, we need to do a couple of things to the symbolication api:

1. add support for sending the code id and code file in the symbolication API request

   Currently, the memoryMap takes a list of two-item lists in the form of:

   [debug id, debug filename]
   

   We should change that to:

   [debug id, debug filename, code id, code filename]
   

   Where the following are all valid:

   1. [debug id, debug filename]
   2. [debug id, debug filename, null, null]
   3. [debug id, debug filename, code id, code filename]

2. We need to change the downloader to add code id and code filename to the download API request when we have both values.

This plan adds on to the existing API in a backwards compatible way, so I don't want to create a v6 of the symbolication API.

Comment 1

[Markus Stange [:mstange]]

API looks good. Does the code id need to have the correct casing or will Tecken worry about giving it the correct casing if it ever interacts with case-sensitive symbol servers? See this issue for background.

(In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #0)

For missing Microsoft symbols, we need the code id and code file to find them.

Not necessarily. You need the code id and code file to get the exe / dll. But for pdb files, the debug id and debug name are enough.
Symbols are contained in the pdb files. You only need the exe / dll if you want to do unwinding or if you want to show assembly code, and Tecken does not do these things today.
So I'm curious what you're planning to do with these exe / dll files in Tecken.

In profiler-get-symbols, I do use the dll / exe for symbols, but only in fallback cases where the pdb is not available. In those cases, I generate synthesized symbols from the list of function addresses in the exe / dll's .pdata section (these synthesized symbols are of the form fun_ab12cd34), and also add the dll exports. But that's a bit of a niche use case.

Comment 2

[Markus Stange [:mstange]]

Oh, I see bug 1768862 now. So the use case for having this information in the request is to be able to trigger other parts of the system to generate .sym files, which requires the code id / code file information because .sym files mix both unwind info and symbol info, and the sym files won't be generated if one of the pieces is missing even though Tecken really only needs the symbol information. All right.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

To clarify, this bug is about getting the wiring in place in Tecken so that the symbolication API gets the required data into the missing symbols report so that Gabriele's Windows system symbol scraper works. This bug is not about adding code to Tecken to generate symbol files or find missing symbol files.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Moving bugs into new Eliot product.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

Per bug #1774004, I'm removing the missing symbols bookkeeping from Tecken. All the information we need for determining which modules we're missing symbols for is in the crash report data and will be in the symbolicated crash ping data in BigQuery. Much like we currently do with the "Weekly report of modules with missing symbols in crash reports" emails, we should use that data to determine missing Microsoft symbols.

Given that, I'm marking this WONTFIX.