Closed Bug 1052007 Opened 7 years ago Closed 7 years ago
Switch to ujson for parsing middleware JSON
Core to the crashstats/models.py is that it parses the JSON coming from the middleware (only if the `expect_json` parameter is True). It does this a lot and it's using the built in json module.
I ran the following code to measure how much time the SocorroCommon class spends on parsing JSON. I used the following code: https://gist.github.com/3dc9b066c154d13ba7c7 Summing it:: peterbe@mpb:/tmp$ python dummy2.py parse.json.log 127 datapoints Total bytes 5644488 5512 Kb Total time 1522.21 seconds peterbe@mpb:/tmp$ python dummy2.py parse.ujson.log 127 datapoints Total bytes 5644488 5512 Kb Total time 44.54 seconds So if we switch to ujson we're saving time 34 times over.
Assignee: nobody → peterbe
Status: NEW → ASSIGNED
Commits pushed to master at https://github.com/mozilla/socorro https://github.com/mozilla/socorro/commit/0a9058cad324bec141bf99d95df7379854e51e2a fixes bug 1052007 - Switch to ujson for parsing middleware JSON https://github.com/mozilla/socorro/commit/3acc5b3e2d6fc51f5b856c3743d473ac6f44dbbb Merge pull request #2283 from peterbe/bug-1052007-switch-to-ujson-for-parsing-middleware-json fixes bug 1052007 - Switch to ujson for parsing middleware JSON
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.