867315 - modify processor to process raven.js crashes

Status: RESOLVED FIXED

(Socorro Graveyard :: Bixie, defect)

Description

[Lonnen :lonnen]

Once we know what raven.js crashes look like, we can modify the processors to write them down and forward them to another error reporting instance.

Comment 1

[K Lars Lohn [:lars] [:klohn]]

The processor just takes a payload of crash data (mapping of data plus any associated blobs) transforms it and then saves its somewhere.  

Since we don't know precisely how we're going to transform the data, and the transformations are likely to evolve rapidly, I suggest that we use a processor piece based entirely on the rewrite rule system.  To do this, we need to make a processor object with just one public method, "convert_raw_crash_to_processed_crash" that accepts a raw crash mapping (and associated blobs) and returns the processed crash.  The implementation of this class would just be load the rules  in the constructor, and a single line "apply rules" in the body of the aforementioned public method.  There'd be very little else to do.

For the actual conversion of a crash from raw to processed, write a function that accepts a crash mapping (plus associated blobs).  If were going to have the concept of a processed crash, pass that in too.  Write code that will do something with the crash data and write it back to the processed mapping, or modify the original crash mapping.

Write as many functions in that form as needed to accomplish whatever "processing" is needed for a raven.js crash.  

List these functions in the rewrite rules table within postgres.  The processor obect will read the function list in its constuctor, dynamically load them and then apply them to the crashes.  As our concept of processing a raven crash changes, just add or delete entries in the postgres rewrite rule table. 

Crash storage systems:  this is where we're likely need some work.  Where do we save these crashes?  HBase?  Postgres?  We will have to create CrashStorage classes for these external resources.

Comment 2

[Lonnen :lonnen]

Processing means decoding and generating a signature.

Comment 3

[Lonnen :lonnen]

Did not mean to reply yet.

The breakdown of the query and the payload is here: https://gist.github.com/lonnen/dafb5fdf8611201572f1

I imagined we'd want to write things down in HBase and then Postgres. We've got a basic schema for it in Postgres. We may also want file system. I don't think HBase will be necessary for developer deployments (collecting for only a few products).

Comment 4

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/3e0d92e55a1e69ba45d0ce41487b33b6a90520e5
Merge pull request #1304 from twobraids/bixie_processor

Fixes Bug 867315 - Bixie processor