Store native debug symbols for Linux as .so.dbg and not as .so.dbg.gz
Categories
(Tecken :: General, task)
Tracking
(Not tracked)
People
(Reporter: mstange, Assigned: mstange)
References
(Depends on 1 open bug, Blocks 1 open bug)
Details
Attachments
(3 files)
For native debug symbols on Linux, we currently compress them to .dbg.so.gz before stuffing them into a zip to upload them to the symbol server.
Then they get served at URLs like the following:
https://symbols.mozilla.org/libxul.so/47F36A5F2226471CD0138DC4623A044F0/libxul.so.dbg.gz
Could we serve them without the .gz extension? We can still use Content-Encoding: gzip for them, if the requesting client indicates support for it with the Accept-Encoding header.
Then they "look" uncompressed but still get transmitted in compressed form.
I think this is what we do for the .sym files - we upload them as .sym (in the zip), and they are served with a .sym extension, but they use Content-Encoding: gzip.
Then, as the next step, we can move them to URLs which are compatible with the debuginfod protocol:
https://symbols.mozilla.org/buildid/5f6af34726221c47d0138dc4623a044fccf0733f/debuginfo
The debuginfod protocol requires the downloaded files to be uncompressed .so.dbg files.
Updated•2 years ago
|
Assignee | ||
Comment 1•2 years ago
|
||
exist_ok was added in Python 3.2. It is already used by uploadsymbols.py.
Updated•2 years ago
|
Assignee | ||
Comment 2•2 years ago
|
||
Depends on D154292
Assignee | ||
Comment 3•2 years ago
|
||
In the past, it was stored at /<debug_name>/<debug_id>/<debug_name>.so.gz.
Example:
Before: https://symbols.mozilla.org/libxul.so/47F36A5F2226471CD0138DC4623A044F0/libxul.so.dbg.gz
After: https://symbols.mozilla.org/buildid/5f6af34726221c47d0138dc4623a044fccf0733f/debuginfo
Depends on D154293
Assignee | ||
Comment 4•2 years ago
|
||
Hmm, so the upload-symbols job completed successfully in the try push, but I can't find the files on the server.
Does Tecken have another file name allow list that I'm running into?
For example, this file exists: https://symbols.mozilla.org/try/ssltunnel/B6D4DCC11470DC818FF28B66C95000600/ssltunnel.sym
But this file does not: https://symbols.mozilla.org/try/buildid/c1dcd4b6701481dc8ff28b66c9500060adddf31d/debuginfo
Both were present in the same .zip file.
Comment 5•2 years ago
|
||
The second item doesn't match the zip file structure, so my guess is that Tecken is dropping those files. I'd need to check the code to be sure. I won't get to that for a bit. Keeping the needinfo until I do.
Assignee | ||
Comment 6•2 years ago
|
||
Ah, I think I've found the explanation:
DOWNLOAD_FILE_EXTENSIONS_ALLOWED = _config(
"DOWNLOAD_FILE_EXTENSIONS_ALLOWED",
default=".sym,.dl_,.ex_,.pd_,.dbg.gz,.tar.bz2",
parser=ListOf(str),
doc=(
"A list of file extensions that if a file is NOT one of these extensions "
"we can immediately return 404 and not bother to process for anything "
"else.\n\n"
"It's case sensitive and has to be lower case. As a way to get marginal "
"optimization of this, make sure '.sym' is first in the list since it's "
"the most common."
),
)
The files are probably on the server, but the URLs don't serve the redirect because they take the early-out-404 path.
Assignee | ||
Comment 7•2 years ago
|
||
I've filed bug 1784290 to take care of that.
Description
•