1776868 - create MissingProcessedCrash API

Status: RESOLVED FIXED

(Socorro :: Webapp, enhancement, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Socorro tries to catch situations where crash reports were collected (a raw crash was created), but never processed (a processed crash was not created and/or a document was not indexed in elasticsearch).

There's a verifyprocessed task that runs every day at 4:00am that goes through the previous day's raw crashes in AWS S3 and checks for a corresponding processed crash and document in elasticsearch. It creates a missing processed crash record for anything missing.

There's a updatemissing task that runs every day at 3:00am that goes through all the missing processed crash records and checks to see if there's a processed crash and a document in elasticsearch and if there is, it updates the missing processed crash record to say it's been processed.

There's a Missing Processed Crash admin page that lets an admin manually reprocess crash reports that haven't been processed. It's great for small numbers of crash reports. It's terrible for reprocessing, say, 10k crash reports.

We've got a bunch of command line tools for building scripts with. If we build a MissingProcessedCrash API that lets us get a list of them, then we could easily script reprocessing for large numbers of crash reports.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Grabbing this to do now because I'm in exactly this situation and I don't want to click through the admin page a million times today.

Also, we might want to add an item to the runbook for this.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6105: bug 1776868: implement MissingProcessedCrash API

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6105: "bug 1776868: implement MissingProcessedCrash API" in 3435c5b.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

I pushed this out in bug #1777242. Marking as FIXED.