466022 - Provide a way to download the associated minidump file

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Jeff Muizelaar [:jrmuizel]]

Making the minidump file available will allow them to be opened in visual studio and examined further. It will also allow them to be converted to core files for use on other platforms.

Comment 1

[(not currently active) Ted Mielczarek]

I don't think we have the storage for this. Each minidump is only ~50k, but we process tens of thousands of crashes a day. It's unfortunate, but it's reality. There was a bug filed (that I can't find, maybe it got closed) about saving a few minidumps per topcrash signature, which still might be a decent way to go.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

Even keeping all the minidumps from the last month would be substantially more useful than not having them at all. If we haven't seen a particular crash in the last month it's probably not _that_ important. 

Still, a years worth of crashes @ 35k per day is only about 600GB which really isn't that much.

Comment 3

[James May [:fowl2]]

Is there a possibility for a (raw) minidump to contain potentially private information? I know that Microsoft's reporting program makes particular mention at various stages about how dumps are encrypted and only allowed to be seen by a small group of people.

Comment 4

[(not currently active) Ted Mielczarek]

Yes, it contains the entire stack for all threads in the process. If we did implement anything like this, it would be limited to authenticated trusted users. Minidumps are currently submitted over SSL and deleted as soon as they are processed.

Comment 5

[Michael Morgan [:morgamic]]

Once we have the admin-panel up, we should provide a way for authorized users to grab the associated mini-dump.

Comment 6

[Michael Morgan [:morgamic]]

-> Ryan for 1.2.  Lars, Frank; we'll need to work with IT and Ryan to determine how to generate these links and offer these files to secure users with the right LDAP permissions.

Comment 7

[David Mandelin [:dmandelin]]

Attachment: Python script

For now, here is a Python 3.1 script that will download a crash by UUID, assuming you have access to the minidump storage machine. Basic usage:

  python3.1 getdump.py 93a2e5b4-ae58-4684-9751-519232091115

Run 'python3.1 getdump.py -h' for options.

Comment 8

[Michael Morgan [:morgamic]]

Aravind - the hard part is to start saving this data so we can pull it later for authenticated users. From there it's pretty easy to grab it.

Comment 9

[Aravind Gottipati [:aravind]]

@Morgamic: so.. we are currently storing the raw dumps already (only for a week).

There is already a bug in place discussing how long they should be stored for (526345).  Is there anything else you guys need from me on this bug?

Comment 10

[Aravind Gottipati [:aravind]]

The dumps should be requested at https://crash-stats.mozilla.com/rawdumps/uuid.json and https://crash-stats.mozilla.com/rawdumps/uuid.dump in production.  In staging that would be https://crash-stats.stage.mozilla.com/rawdumps/uuid.dump and https://crash-stats.stage.mozilla.com/rawdumps/uuid.json.

Comment 11

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Attachment: Patch 1 for 466022

Raw dumps will no longer be available via the file system.  This patch provides urls for the logged-in developer to fetch either the .dump file and .json file containing the raw dump.

Per Avarind, Apache will be configured to prompt the user for their LDAP credentials when accessing either of these files.  A separate ticket will be created for this after the patch is approved.

Comment 12

[Austin King [:ozten]]

Comment on attachment 414730 [details] [diff] [review]
Patch 1 for 466022

Very cool. Nice touch with the raw_dump_availability.

Comment 13

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Committing.

Updated docs at http://code.google.com/p/socorro/wiki/SocorroUpgrade

Will need config file on stage updated.  See #531872.

==

Sending        application/config/application.php-dist
Sending        application/controllers/report.php
Sending        application/models/report.php
Sending        application/views/report/index.php
Transmitting file data ....
Committed revision r1520.