Processor should crash gracefully when minidump_stackwalk fails

RESOLVED FIXED

Status

RESOLVED FIXED
11 years ago
7 years ago

People

(Reporter: morgamic, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

11 years ago
Created attachment 307603 [details]
example crash

When minidump_stackwalk fails horribly, the processor enters a zombie state:
1) modules and frames inserts fail in succession
2) the report id from createReport (part of our failed mutual exclusion noted in bug 420809)
3) a zombie report exists where the processor continually thinks it's been processed, but it contains no frames or modules (and thus is essentially useless)

To fix:
1) enable transactions for report, frames, modules and dump creation
2) rollback if minidump_stackwalk fails miserably at any point during the processing of a single report
(Reporter)

Comment 1

11 years ago
I punted on transactions.  Fix on trunk in revision 313 adds a check so we don't try to execute insertions for module records that never get populated by the breakpad stack processor.

The errors in the attachment resulted in a null modules list, which in turned caused the exception.

The exception caused .dump/.json cleanup to fail, and the monitor doesn't recover gracefully from this.

Once 420809 is fixed, we can explore better methods to handle this type of scenario like:
* never committing a report record unless a signature exists (include crash signature with initial insert)
* executing some sort of rollback if frames, modules or especially dumps table inserts fail after report insertion using SA transactions (though there may be speed concerns there)

Feel like I'm talking to myself!  Resolving.
Status: NEW → RESOLVED
Last Resolved: 11 years ago
Resolution: --- → FIXED
(Assignee)

Updated

7 years ago
Component: Socorro → General
Product: Webtools → Socorro
You need to log in before you can comment on or make changes to this bug.