2006338 - We provide URLs to gzip-compressed generated source files in our PDB srcsrv stream, Microsoft debuggers won't recognize the format

Status: NEW

(Toolkit :: Crash Reporting, defect)

Description

[Yannis Juglaret [:yannis]]

Mozilla PDBs embed a srcsrv stream to let debuggers retrieve the source files through HTTP. Currently, each source file will be retrieved from one of HG_TARGET, S3_TARGET, RUST_GITHUB_TARGET or FIREFOX_GITHUB_TARGET. In that list, S3_TARGET is where we upload the files generated while building, that don't have a direct match in the source code. An example of such an URL is:

https://gecko-generated-sources.s3.amazonaws.com/ce9aa491dc53ddb594986234d30888c7fb4b127f65581786d981ef140d1ed10544105e72386fa3b57adeb1fac8107d66ee1125c5503a6904c5fe0b344003dfda/ipc/ipdl/DOMTypes.cpp

Following this URL does end up opening the correct source code in Firefox. Looking at the request in Developer Tools, Firefox uses a Accept-Encoding: gzip, deflate, br, zstd that probably explains the correct behavior.

However, when Microsoft debuggers retrieve the file, they either refuse to show the contents (Visual Studio) or show garbled file contents (WinDbg). Basically, they want to "show" the gzipped data and don't understand that it is gzipped.

I can get a similar result with curl, where the tool only retrieves the data correctly if using --compressed. Otherwise, it retrieves gzipped data:

$ curl https://gecko-generated-sources.s3.amazonaws.com/ce9aa491dc53ddb594986234d30888c7fb4b127f65581786d981ef140d1ed10544105e72386fa3b57adeb1fac8107d66ee1125c5503a6904c5fe0b344003dfda/ipc/ipdl/DOMTypes.cpp -o DOMTypes.cpp
$ file DOMTypes.cpp
DOMTypes.cpp: gzip compressed data, last modified: Tue Dec 16 06:11:40 2025, max compression, original size modulo 2^32 176761

$ curl --compressed https://gecko-generated-sources.s3.amazonaws.com/ce9aa491dc53ddb594986234d30888c7fb4b127f65581786d981ef140d1ed10544105e72386fa3b57adeb1fac8107d66ee1125c5503a6904c5fe0b344003dfda/ipc/ipdl/DOMTypes.cpp -o DOMTypes.cpp
$ file DOMTypes.cpp
DOMTypes.cpp: C++ source, ASCII text

This makes me wonder if perhaps we are currently returning gzipped data unconditionnally, instead of checking the value of Accept-Encoding.

Comment 1

[Yannis Juglaret [:yannis]]

For WinDbg, the workaround mentioned in bug 2006283 comment 1 also works around this gzip issue, thanks to the --compressed option passed to curl.exe.

Comment 2

[Gabriele Svelto [:gsvelto]]

(In reply to Yannis Juglaret [:yannis] from comment #0)

This makes me wonder if perhaps we are currently returning gzipped data unconditionnally, instead of checking the value of Accept-Encoding.

If these files are on S3 then that's probably S3's fault.

Comment 3

[Yannis Juglaret [:yannis]]

(In reply to Gabriele Svelto [:gsvelto] from comment #2)

If these files are on S3 then that's probably S3's fault.

That is maybe configurable? Do you know what component or team at Mozilla is responsible for configuring this server?

Comment 4

[Gabriele Svelto [:gsvelto]]

I don't, Will do you know who was responsible for this?

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I know about symbols files, but I don't know about source files. Maybe it's part of the build pipeline?

Comment 6

[Gabriele Svelto [:gsvelto]]

Could be. Thanks, I'm gonna ask around.

Comment 7

[Gabriele Svelto [:gsvelto]]

The SRE team confirmed that S3 buckets will ignore Accept-Encoding and return whatever was the Content-Encoding type a file was uploaded with. We can't fix it on AWS' side unless we put something in front of it dealing with the encoding.