475165 - Improve compression used on pdb files in symbol store

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

Attachment: Use increased compression settings when calling makecab

As mentioned in bug 385792 comment #14 and on, makecab supports higher compression settings than default. The attached one-liner adds those settings. Using those settings on the xul.pdb Ted created for bug 385792, the resulting xul.pd_ file comes out to 12.3MB instead of 18.4MB with the default settings.

I haven't tested this patch or the resulting pdb file since I don't build crashreporter for my own builds, so someone will want to test it before checking in. However, being that they've been part of the CAB format for over 10 years, I don't foresee any problems in using them.

I don't believe that removing the .exe from the makecab call will be problematic, but if it does, feel free to add it back. I took it off to save a little space.

Comment 1

[(not currently active) Ted Mielczarek]

Comment on attachment 358608 [details] [diff] [review]
Use increased compression settings when calling makecab

-        success = call(["makecab.exe", full_path, compressed_file],
+        success = call(["makecab /D CompressionType=LZX /D CompressionMemory=21", full_path, compressed_file],
                        stdout=open("NUL:","w"), stderr=STDOUT)

Not quite right, note that each entry of that list is a parameter to the commandline. You want something like:
["makecab.exe", "/D", ... ]

Comment 2

[Yuhong Bao]

BTW, LZX was added to the cab format when Jonathan Forbes went to work for MS back in 1997.

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Attachment: Use increased compression settings when calling makecab, try 2

Let's see if this is more to your liking. As was the case before, this is completely untested by me, so I recommend you try it out before committing :-)

Comment 4

[(not currently active) Ted Mielczarek]

Comment on attachment 360182 [details] [diff] [review]
Use increased compression settings when calling makecab, try 2

Looks good, I'll try to find some time to give it a go before I push it. :)

Comment 5

[Ryan VanderMeulen [:RyanVM]]

Find any time to give it a go yet? 33% wins are nice :)

Comment 6

[(not currently active) Ted Mielczarek]

No, and I'm unlikely to. If you want this, you'll have to test it yourself or get someone else to.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

K Ted, it's been a year now. I don't have the ability to test this. If it's ever going to get in, I'm going to need some help. If you can't test it, can you recommend someone else who can?

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Attachment: Updated to tip

Carrying over ted's previous r+. I've tested this patch now using make buildsymbols. PDB compression goes fine.

firefox-4.0b2pre.en-US.win32.crashreporter-symbols-full.zip
Without: 43868KB
With: 33678KB

A 23% reduction in size!

Comment 9

[Dão Gottwald [:dao]]

http://hg.mozilla.org/mozilla-central/rev/0f3542a15b5b

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Ted, do we want this on 1.9.1 and 1.9.2 as well? Should be safe enough.

Comment 11

[(not currently active) Ted Mielczarek]

Feel free to nominate and convince drivers. The symbol server is widely used by both developers and testers.