Closed Bug 1052007 Opened 7 years ago Closed 7 years ago

Switch to ujson for parsing middleware JSON

Categories

(Socorro :: Webapp, task)

x86
macOS
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: peterbe, Assigned: peterbe)

Details

Core to the crashstats/models.py is that it parses the JSON coming from the middleware (only if the `expect_json` parameter is True).

It does this a lot and it's using the built in json module.
I ran the following code to measure how much time the SocorroCommon class spends on parsing JSON.
I used the following code: https://gist.github.com/3dc9b066c154d13ba7c7

Summing it::

peterbe@mpb:/tmp$ python dummy2.py parse.json.log
127 datapoints
Total bytes 5644488 5512 Kb
Total time 1522.21 seconds
peterbe@mpb:/tmp$ python dummy2.py parse.ujson.log
127 datapoints
Total bytes 5644488 5512 Kb
Total time 44.54 seconds


So if we switch to ujson we're saving time 34 times over.
Assignee: nobody → peterbe
Status: NEW → ASSIGNED
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/0a9058cad324bec141bf99d95df7379854e51e2a
fixes bug 1052007 - Switch to ujson for parsing middleware JSON

https://github.com/mozilla/socorro/commit/3acc5b3e2d6fc51f5b856c3743d473ac6f44dbbb
Merge pull request #2283 from peterbe/bug-1052007-switch-to-ujson-for-parsing-middleware-json

fixes bug 1052007 - Switch to ujson for parsing middleware JSON
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → 98
You need to log in before you can comment on or make changes to this bug.