385792 - compress pdb files in symbol store

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, defect)

Description

[(not currently active) Ted Mielczarek]

The MS Symbol Server client supports compressed pdb files in a symbol store.  The compression must be done using compress.exe (or its equivalent).  The compressed pdb files must use a .pd_ extension.

This will help with symbol download speed since compress gives ~50% reduction in pdb size, and since xul.pdb is >70Mb, that's a big win.  Should also lower server load for the same reason.

This will be a little tricky to implement, only that compress.exe doesn't come standard with Windows, and installing the reskit puts it in Program Files somewhere, so MozillaBuild won't pick it up in PATH.  There is a GPL implementation of the tool that we might consider rolling into MozillaBuild here:
http://gnuwin32.sourceforge.net/packages/mscompress.htm

The only downside to this tool is that its commandline options are pretty weak compared to compress.exe.  I might do a quick hack to make it support -R (rename the file after compressing it, so file.exe -> file.ex_) like compress.exe.

Comment 1

[(not currently active) Ted Mielczarek]

Just as a point of information, I tried manually compressing firefox.pdb with compress.exe (to create firefox.pd_) and put it on a test symbol server. symchk.exe was unable to successfully download it. Sniffing the HTTP traffic shows that the HTTP request goes through fine, it just fails to use it for some reason.

Comment 2

[Yuhong Bao]

You can try makecab.exe, which ships with Windows. Just typing makecab xul.pdb will produce xul.pd_, which is much smaller

Comment 3

[Yuhong Bao]

More on makecab.exe:
http://msdn.microsoft.com/en-us/library/bb417343.aspx

Comment 4

[(not currently active) Ted Mielczarek]

Yuhong: I might give that a try, but per comment 1 I tried using compress.exe to do this and couldn't get it to work with a debugger.

Comment 5

[Yuhong Bao]

"but per comment 1 I tried using compress.exe
to do this and couldn't get it to work with a debugger."
That is why I am asking you to try a different utility, in fact, I was able to makecab xul.pdb on my symbol cache and WinDbg successfully decompressed it.

Comment 6

[(not currently active) Ted Mielczarek]

Thanks, I'll give this a shot!

Comment 7

[Yuhong Bao]

"Thanks, I'll give this a shot!"
Please do, because pdbs this large are annoying, especially because the majority of apps that uses the symbol server provide no download progress, only WinDbg does, and I had several occasions where I ended up killing the process, leaving a corrupt pdb on the hard drive.

Comment 8

[Yuhong Bao]

symstore also has a /compress option.

Comment 9

[(not currently active) Ted Mielczarek]

We're not using the actual symstore script. We create the symbol store as a side effect of generating the Breakpad symbols in this script:
http://mxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/tools/symbolstore.py

Comment 10

[(not currently active) Ted Mielczarek]

This appears to work. Test build at:
http://mavra.perilith.com/~luser/firefox-3.2a1pre.en-US.win32.zip

With symbols available at:
http://mavra.perilith.com/~luser/symbols/

Comment 11

[(not currently active) Ted Mielczarek]

Attachment: use makecab to compress symbols

Pretty simple patch.

Comment 12

[Yuhong Bao]

Works, and file is much smaller:
SYMSRV:  xul.pdb from http://mavra.perilith.com/~luser/symbols/: 19397600 bytes
- copied
DBGHELP: xul - private symbols & lines
         c:\downloads\symbols\xul.pdb\4F20D7FCA8484AD7A8E5B68BF6D0C3091\xul.pdb

Comment 13

[(not currently active) Ted Mielczarek]

Pushed to m-c:
http://hg.mozilla.org/mozilla-central/rev/db2cabebef39

Comment 14

[Ryan VanderMeulen [:RyanVM]]

When I add the following command line arguments to makecab, I get xul.pd_ down to 12.3MB.
/D CompressionType=LZX /D CompressionMemory=21

Comment 15

[(not currently active) Ted Mielczarek]

Are those standard options included with all versions of makecab? (Just a sanity check!) Also, care to attach a patch?

Comment 16

[Ben Turner (not reading bugmail, use the needinfo flag!)]

And more importantly can windbg and vstudio still download and decompress them?

Comment 17

[Ryan VanderMeulen [:RyanVM]]

They've been standard options in the Microsoft CAB SDK since version 1.0. I've got a paper dated March of 1997 documenting those settings. So yes, it's been around for awhile.

There's no reason windbg and vstudio shouldn't be able to work with them. They're still standard CAB files. All those settings do is use LZX compression (a standard CAB feature as noted above) and a larger dictionary size setting. I can attach a patch, but I won't be easily able to test the resulting files.

Comment 18

[Yuhong Bao]

Yep, I posted a link to MSDN that has the makecab info.

Comment 19

[Yuhong Bao]

"So yes, it's been around for awhile."
Yep, LZX was added to the CAB format when Forbes went to work for MS back in around 1997.

Comment 20

[(not currently active) Ted Mielczarek]

Comment on attachment 357520 [details] [diff] [review]
use makecab to compress symbols

We should take this on branch, it makes using the symbol server a lot less painful. It's a very small patch which only impacts the Windows symbol server portion of symbol generation.

Comment 21

[Yuhong Bao]

Yep, I agree, because I just recently felt the pain you are talking about with a release in the 1.9.0 branch.

Comment 22

[Mike Shaver (:shaver emeritus)]

Comment on attachment 357520 [details] [diff] [review]
use makecab to compress symbols

Agreed, makes the process of getting stacks from helpful reporters a ton more pleasant. a191=shaver

Comment 23

[(not currently active) Ted Mielczarek]

Pushed to 1.9.1:
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/712b74ad6d37

Comment 24

[Yuhong Bao]

In fact, I am now in the habit of running SYMCHK on the Firefox directory each time I install a new release of Firefox exactly because of this. That in fact was how I found bug 489719.