Send code_id and code_file as query strings when querying symbols



9 months ago
8 months ago


(Reporter: peterbe, Assigned: ted)


(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)




9 months ago
In we're ultimately trying to remove all dealings with missing symbols from the socorro processor. 

Instead, the plan is to start querying from too which logs all 404s and thus solves this problem of ultimately producing a CSV file of symbols that were missing. 

Because stackwalker needs to emit slightly more information, about which symbols couldn't be looked up, we need to now instead pack that into the URL as query string parameters. 

As the stackwalker symbolicates it knows the name of the file in S3 (e.g. 'foo.pdb/946C0C63132015DD88/foo.sym`) but it also has a code_file and code_id. These need to be sent as query string paramters. I.e. `GET foo.pdb/946C0C63132015DD88/foo.sym?code_file=CODE_FILE&code_id=CODE_ID`.

Once the stackwalker has this functionality, we can through configuration in Socorro Processor, add a third URL to the invocation of stackwalker.

Comment 1

9 months ago
Dare I assign this to you? 

You might notice from my language above that I'm not entirely sure what this entails for the innards of stackwalker. But I'm eager to learn and to help.

By the way, stackwalker is currently invoked with two `--symbols-url=...` flags. [0]
Considering that soon will work the same as we could consider changing Socorro to use directly (the first of the two flags) for the public symbols. Theoretically we're pretty certain the code copes with redirects through curl. 

I don't know what the performance and availability difference is going to be yet. 

Flags: needinfo?(ted)


9 months ago
Assignee: nobody → ted
Flags: needinfo?(ted)

Comment 3

8 months ago
Commit pushed to master at
bug 1363177 - send code_id and code_file in query string when fetching symbols via HTTP (#3775)

This will allow the symbol server to store the info necessary for missing
symbols so we can fetch them from the Microsoft symbol server.

Comment 4

8 months ago
One last thing I'd like to do...
Once this goes into stage the URL that's hitting our S3 buckets is going to change. I should be able to see that if I enabled S3 bucket logging temporarily and inspect the URLs that it requests.

Comment 5

8 months ago
I did some logging on the S3 symbol buckets, downloaded some of those logs and lo and behold there's a bunch of GET requests that have the query string. E.g.

GET /org.mozilla.crash-stats.symbols-public/v1/sysfer.pdb/28695B7D959F48ACB087879FD7CDC4211/sysfer.sym?code_file=sysfer.dll&code_id=587B937A7b000

Also, I looked at a bunch of random rows in `missing_symbols_20170515 WHERE date_processed='2017-05-16'` and the data there looks equally sane. I.e. no '?code_file' within the 'debug_file' column.
Last Resolved: 8 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.