Closed Bug 1668317 Opened 4 months ago Closed 4 months ago

at some point in the last two months, dlls stopped being compressed

Categories

(Firefox Build System :: General, defect, P1)

defect

Tracking

(firefox-esr78 unaffected, firefox81 fixed, firefox82 fixed, firefox83 fixed)

RESOLVED FIXED
Tracking Status
firefox-esr78 --- unaffected
firefox81 --- fixed
firefox82 --- fixed
firefox83 --- fixed

People

(Reporter: willkg, Assigned: glandium)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

At some point, the build system extracts symbols from binaries it's just built and then later uploads them to Tecken. For Windows builds, it runs makecab on files to compress them. This happens before uploading.

At some point in the last 3 months (I can spend more time to figure out exactly when things changed), .dll files stopped being compressed. I did some spot checking and found these two uploads--somewhere between those two, things changed.

This upload from July 8th, 2020 has compressed dlls:

https://symbols.mozilla.org/uploads/upload/145567

It has this record:

v1/xul.dll/5F064109699B000/xul.dl_  41.7 MB

This upload from August 7th, 2020 does not have compressed dlls:

https://symbols.mozilla.org/uploads/upload/165577

It has this record:

v1/xul.dll/5F2D86426EB9000/xul.dll    109.9 MB

The consequence of this is that VS can no longer download dlls. It's trying to download:

https://symbols.mozilla.org/xul.dll/5F517CAB659C000/xul.dll

But "dll" is not in the allowed extensions list, so it gets back a 404 (ignored) message.

This bug covers fixing something so VS works again.

cc:ing everyone involved.

Braindump:

I think (but I'm not sure because I don't work on any of that stuff) the work done in this bug caused the problem:

https://bugzilla.mozilla.org/show_bug.cgi?id=1654994

That moved where makecab was being called. Maybe we stopped calling it on dlls?

Tecken has a setting for "allowed extensions" which it'll let people download. "dl_" is in the list, but "dll" is not. So the dll files are in the symbols bucket, but because they're not compressed, they have "dll" as an extension and not "dl_" and thus requesting them yields the 404 (ignored) message. That's what Jens and Simon are hitting.

Bug #1635150 covers figuring out whether Tecken can handle uploads of non-breakpad symbols files that are uncompressed. I didn't have time to look into that.

Options I can think of:

  1. We change the setting in Tecken to also allow downloading of "dll". I don't know how uncompressed files affects storage costs. I don't know if there are other files that need to be allowed.
  2. We fix upload_symbols to call makecab on dll files. https://searchfox.org/mozilla-central/source/toolkit/crashreporter/tools/upload_symbols.py#135
  3. Some third option?
  4. All of the above?

Tentatively setting regressed by 1654994 and ni to glandium:. Thanks!

Flags: needinfo?(mh+mozilla)
Regressed by: 1654994
Assignee: nobody → mh+mozilla
Status: NEW → ASSIGNED
Flags: needinfo?(mh+mozilla)
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/37a2334aacb4
Also compress .exe and .dll files for the symbol server. r=firefox-build-system-reviewers,mhentges,rstewart DONTBUILD
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED

If I understand what happened correctly, this will fix the nightly channel, but not the beta channel or release channel. Do both those channels also have the issue? If so, can this get uplifted?

Component: Symbols → General
Product: Socorro → Firefox Build System

Comment on attachment 9178913 [details]
Bug 1668317 - Also compress .exe and .dll files for the symbol server.

Beta/Release Uplift Approval Request

  • User impact if declined: Visual Studio can't load the symbols for crash reports.
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): It doesn't affect Firefox directly, and has been verified to work.
  • String changes made/needed: n/a
Attachment #9178913 - Flags: approval-mozilla-release?
Attachment #9178913 - Flags: approval-mozilla-beta?
See Also: → 1668795

Comment on attachment 9178913 [details]
Bug 1668317 - Also compress .exe and .dll files for the symbol server.

approved for 82.0b8

Attachment #9178913 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

Comment on attachment 9178913 [details]
Bug 1668317 - Also compress .exe and .dll files for the symbol server.

Approved for 81.0.2

Attachment #9178913 - Flags: approval-mozilla-release? → approval-mozilla-release+
You need to log in before you can comment on or make changes to this bug.