at some point in the last two months, dlls stopped being compressed
Categories
(Firefox Build System :: General, defect, P1)
Tracking
(firefox-esr78 unaffected, firefox81 fixed, firefox82 fixed, firefox83 fixed)
Tracking | Status | |
---|---|---|
firefox-esr78 | --- | unaffected |
firefox81 | --- | fixed |
firefox82 | --- | fixed |
firefox83 | --- | fixed |
People
(Reporter: willkg, Assigned: glandium)
References
(Regression)
Details
(Keywords: regression)
Attachments
(1 file)
47 bytes,
text/x-phabricator-request
|
jcristau
:
approval-mozilla-beta+
pascalc
:
approval-mozilla-release+
|
Details | Review |
At some point, the build system extracts symbols from binaries it's just built and then later uploads them to Tecken. For Windows builds, it runs makecab on files to compress them. This happens before uploading.
At some point in the last 3 months (I can spend more time to figure out exactly when things changed), .dll files stopped being compressed. I did some spot checking and found these two uploads--somewhere between those two, things changed.
This upload from July 8th, 2020 has compressed dlls:
https://symbols.mozilla.org/uploads/upload/145567
It has this record:
v1/xul.dll/5F064109699B000/xul.dl_ 41.7 MB
This upload from August 7th, 2020 does not have compressed dlls:
https://symbols.mozilla.org/uploads/upload/165577
It has this record:
v1/xul.dll/5F2D86426EB9000/xul.dll 109.9 MB
The consequence of this is that VS can no longer download dlls. It's trying to download:
https://symbols.mozilla.org/xul.dll/5F517CAB659C000/xul.dll
But "dll" is not in the allowed extensions list, so it gets back a 404 (ignored) message.
This bug covers fixing something so VS works again.
Reporter | ||
Comment 1•4 months ago
|
||
cc:ing everyone involved.
Braindump:
I think (but I'm not sure because I don't work on any of that stuff) the work done in this bug caused the problem:
https://bugzilla.mozilla.org/show_bug.cgi?id=1654994
That moved where makecab was being called. Maybe we stopped calling it on dlls?
Tecken has a setting for "allowed extensions" which it'll let people download. "dl_" is in the list, but "dll" is not. So the dll files are in the symbols bucket, but because they're not compressed, they have "dll" as an extension and not "dl_" and thus requesting them yields the 404 (ignored) message. That's what Jens and Simon are hitting.
Bug #1635150 covers figuring out whether Tecken can handle uploads of non-breakpad symbols files that are uncompressed. I didn't have time to look into that.
Options I can think of:
- We change the setting in Tecken to also allow downloading of "dll". I don't know how uncompressed files affects storage costs. I don't know if there are other files that need to be allowed.
- We fix upload_symbols to call makecab on dll files. https://searchfox.org/mozilla-central/source/toolkit/crashreporter/tools/upload_symbols.py#135
- Some third option?
- All of the above?
Comment 2•4 months ago
|
||
Tentatively setting regressed by 1654994 and ni to glandium:. Thanks!
Assignee | ||
Comment 3•4 months ago
|
||
Updated•4 months ago
|
Assignee | ||
Updated•4 months ago
|
Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/37a2334aacb4 Also compress .exe and .dll files for the symbol server. r=firefox-build-system-reviewers,mhentges,rstewart DONTBUILD
Comment 5•4 months ago
|
||
bugherder |
Reporter | ||
Comment 6•4 months ago
|
||
If I understand what happened correctly, this will fix the nightly channel, but not the beta channel or release channel. Do both those channels also have the issue? If so, can this get uplifted?
Updated•4 months ago
|
Assignee | ||
Comment 7•4 months ago
|
||
I can confirm it worked: https://symbols.mozilla.org/xul.dll/5F7B502C6A9B000/xul.dl_
Assignee | ||
Updated•4 months ago
|
Assignee | ||
Comment 8•4 months ago
|
||
Comment on attachment 9178913 [details]
Bug 1668317 - Also compress .exe and .dll files for the symbol server.
Beta/Release Uplift Approval Request
- User impact if declined: Visual Studio can't load the symbols for crash reports.
- Is this code covered by automated tests?: No
- Has the fix been verified in Nightly?: Yes
- Needs manual test from QE?: No
- If yes, steps to reproduce:
- List of other uplifts needed: None
- Risk to taking this patch: Low
- Why is the change risky/not risky? (and alternatives if risky): It doesn't affect Firefox directly, and has been verified to work.
- String changes made/needed: n/a
Comment 9•4 months ago
|
||
Comment on attachment 9178913 [details]
Bug 1668317 - Also compress .exe and .dll files for the symbol server.
approved for 82.0b8
Comment 10•4 months ago
|
||
bugherderuplift |
Comment 11•3 months ago
|
||
Comment on attachment 9178913 [details]
Bug 1668317 - Also compress .exe and .dll files for the symbol server.
Approved for 81.0.2
![]() |
||
Comment 12•3 months ago
|
||
bugherderuplift |
Description
•