1052007 - Switch to ujson for parsing middleware JSON

Status: RESOLVED FIXED

(Socorro :: Webapp, task)

Description

[Peter Bengtsson [:peterbe]]

Core to the crashstats/models.py is that it parses the JSON coming from the middleware (only if the `expect_json` parameter is True).

It does this a lot and it's using the built in json module.

Comment 1

[Peter Bengtsson [:peterbe]]

I ran the following code to measure how much time the SocorroCommon class spends on parsing JSON.
I used the following code: https://gist.github.com/3dc9b066c154d13ba7c7

Summing it::

peterbe@mpb:/tmp$ python dummy2.py parse.json.log
127 datapoints
Total bytes 5644488 5512 Kb
Total time 1522.21 seconds
peterbe@mpb:/tmp$ python dummy2.py parse.ujson.log
127 datapoints
Total bytes 5644488 5512 Kb
Total time 44.54 seconds


So if we switch to ujson we're saving time 34 times over.

Comment 2

[Peter Bengtsson [:peterbe]]

PR https://github.com/mozilla/socorro/pull/2283

Comment 3

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/0a9058cad324bec141bf99d95df7379854e51e2a
fixes bug 1052007 - Switch to ujson for parsing middleware JSON

https://github.com/mozilla/socorro/commit/3acc5b3e2d6fc51f5b856c3743d473ac6f44dbbb
Merge pull request #2283 from peterbe/bug-1052007-switch-to-ujson-for-parsing-middleware-json

fixes bug 1052007 - Switch to ujson for parsing middleware JSON