607969 - Convert all text-format symbols on the symbol server to binary format

Status: RESOLVED WONTFIX

(Socorro :: General, task)

Description

[(not currently active) Ted Mielczarek]

In bug 607831, we're going to be changing the format of the symbols that Socorro uses. The new binary format will make processing much faster. However, we'll have to convert all of our existing symbols to the new format. I have a little binary + script that can do this, it will just take some time to convert all the symbols we have on the server, since it has to parse every single symbol file and write out the binary conversion. Also, this will increase our disk usage on the symbol server by quite a bit until we switch over Socorro to use the new format, and delete the old text-format symbols. It shouldn't double our disk usage, since a large portion of our disk usage on the symbol server is the native-format debugging symbols, which will be untouched here, but it will definitely increase it quite a bit. We've got a little bit of headroom on the symbol mount right now, mostly because aravind bumped it up to 1.3TB:
10.253.0.11:/vol/socorro/symbols
                      1.3T  874G  379G  70% /mnt/netapp/breakpad

but I'm not sure that that will be enough space to store all the converted symbols. We'll need to figure out when we want to deploy this, then give ourselves a window of time to do the conversion, during which we may need to increase the symbol storage to give a margin of safety. After deploying the Socorro update we can then delete the text-format symbols, which should shrink our disk usage back down.

Comment 1

[(not currently active) Ted Mielczarek]

The post upload script on bug 607961 should work fine for the mass-conversion. We can just run it like "ls -1 /path/to/symbols_whatever/*-symbols.txt | xargs /usr/local/bin/post-symbol-upload.py".

Comment 2

[(not currently active) Ted Mielczarek]

WONTFIXing in lieu of bug 789493.