1051984 - Use ujson to serialize JSON

Status: RESOLVED FIXED

(Socorro Graveyard :: Middleware, defect)

Description

[Peter Bengtsson [:peterbe]]

ujson is much faster than the plain json module and we're not using anything fancy when doing the dumps(). 

One interesting, but should be irrelevant, difference is that ujson doesn't bother with readable spaces after commas::

 PLAIN JSON
 {"hits": [{"id": 1041646, "signature": "mozilla::dom::exceptions::JSStackFrame::GetFilename(nsAString_internal&)"}], "total": 1}
 UJSON
 {"hits":[{"id":1041646,"signature":"mozilla::dom::exceptions::JSStackFrame::GetFilename(nsAString_internal&)"}],"total":1}

Comment 1

[Peter Bengtsson [:peterbe]]

I think this only matters on the /crash_data/ endpoint where the payload is big enough for there to be a time difference to make the dump string.

Comment 2

[Peter Bengtsson [:peterbe]]

Another interesting difference is how floating point numbers are serialized. Compare::

PLAIN JSON
{"hits": [["20140802030201", 39.173999999999999], ["20140803030203", 57.847999999999999], ["20140804030205", 21.591000000000001], ["20140805030300", 30.475000000000001], ["20140806030201", 11.906000000000001], ["20140807030202", 25.768000000000001], ["20140807084340", 46.316000000000003], ["20140808030201", 39.737000000000002], ["20140809030202", 40.337000000000003]], "total": 9}
UJSON
{"hits":[["20140802030201",39.174],["20140803030203",57.848],["20140804030205",21.591],["20140805030300",30.475],["20140806030201",11.906],["20140807030202",25.768],["20140807084340",46.316],["20140808030201",39.737],["20140809030202",40.337]],"total":9}

Comment 3

[Peter Bengtsson [:peterbe]]

Attachment: Screenshot 2014-08-11 09.52.59.png

Comparison with outliers that mess up the chart.

Comment 4

[Peter Bengtsson [:peterbe]]

Attachment: Screenshot 2014-08-11 09.54.00.png

Charts without the outliers ruining the comparison :)

Comment 5

[Peter Bengtsson [:peterbe]]

I measured what the middleware was doing for a while and summed it here::

PLAIN JSON
245 datapoints
Total bytes 20442428 19963 Kb
Total time 2.23465 seconds

UJSON
198 datapoints
Total bytes 15677215 15309 Kb
Total time 0.09736 seconds

Basically, the difference really does add up.

Comment 6

[Peter Bengtsson [:peterbe]]

PR https://github.com/mozilla/socorro/pull/2282

Comment 7

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/0231a4c2f0c2e292b0969e62cf64ef23b45b8173
fixes bug 1051984 - Use ujson to serialize JSON

https://github.com/mozilla/socorro/commit/200312d8c2bba4b0f1f08c818a357d7336133455
Merge pull request #2282 from peterbe/bug-1051984-use-ujson-to-serialize-json

fixes bug 1051984 - Use ujson to serialize JSON