898300 - test_run_id is null in DataZilla stored JSON for recent B2G results

Status: RESOLVED FIXED

(Datazilla Graveyard :: Database, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Since around Jul 20, 2013 6:17:06 PM DataZilla results have not been displayed in the UI. On examining the stored JSON, it appears that they are being stored with a null test_run_id value.

Stored JSON from FPS testrun on Jul 20, 2013 7:38:45 AM:
{"json_blob": {"test_machine": {"platform": "Gonk", "osversion": "1.2.0.0-prerelease", "os": "Firefox OS", "name": "48:28:2f:f9:ba:3e", "type": "unagi"}, "testrun": {"date": 1374333168, "suite": "homescreen"}, "results": {"fps": [58.082055252471157, 60.249357890675292, 60.000641973402537, 58.826918804584288, 56.482910341198483, 59.141495135701035, 59.487570550666085, 56.846746553889147, 56.022608540761865, 59.985638681942469, 59.387321149206571, 59.08536554077228, 59.183016272847013, 60.214933253019254, 57.113707329889266, 56.509968418181145, 56.319798689804209, 56.060483220924908, 57.304957556820966, 56.936493631090741, 58.074184304664371, 59.020495027505561, 56.795567787345348, 58.451597229383587, 57.768257411195755, 57.038718887254262, 55.736838929976273, 57.978588690578142, 55.412769700060913, 57.425852065360402]}, "test_build": {"name": "B2G", "build_revision": "a56a09f9b3342e5d5cb05699288913035f4f6e48", "delay": 5.0, "version": "prerelease", "gecko_revision": "7b496f774fb09fe78df31b918880e7ba097ce6b1", "branch": "master", "gaia_revision": "bbad03e30527e1df8261a462e5cefbdc562f0819", "settle_time": 60, "id": "20130720030213", "restart": true, "revision": "bbad03e30527e1df8261a462e5cefbdc562f0819"}}, "date_loaded": 1374333145, "error_flag": "N", "test_run_id": 48050, "processed_flag": "complete", "error_msg": ""}

Stored JSON from FPS testrun on Jul 20, 2013 6:17:06 PM:
{"json_blob": {"test_machine": {"platform": "Gonk", "osversion": "1.2.0.0-prerelease", "os": "Firefox OS", "name": "48:28:2f:f9:ba:3e", "type": "unagi"}, "testrun": {"date": 1374371421, "suite": "homescreen"}, "results": {"fps": [58.079303608270649, 60.170772761825745, 59.954992716977721, 59.175660753878823, 57.329387698540401, 58.821755770777223, 56.739808006768122, 55.170964070811294, 55.500249081241371, 59.128504614444843, 59.164298753903751, 59.398311072642443, 59.179733092569016, 58.549840899816367, 58.202834727676581, 57.691153161558503, 56.113100416838776, 57.58419647550663, 57.644039967397674, 56.791827710059941, 57.083042535695824, 58.931114005607469, 56.608263077640792, 57.947635671768069, 59.486153823507912, 55.919195127930479, 57.009113389173294, 57.960627598086973, 55.835365226936787, 57.307982433354816]}, "test_build": {"name": "B2G", "build_revision": "a56a09f9b3342e5d5cb05699288913035f4f6e48", "delay": 5.0, "version": "prerelease", "gecko_revision": "7b496f774fb09fe78df31b918880e7ba097ce6b1", "branch": "master", "gaia_revision": "52b07818962269a569f61523b1cae0765ab3f1d8", "settle_time": 60, "id": "20130720030213", "restart": true, "revision": "52b07818962269a569f61523b1cae0765ab3f1d8"}}, "date_loaded": 1374371423, "error_flag": "N", "test_run_id": null, "processed_flag": "ready", "error_msg": ""}

Comment 1

[Jonathan Eads ( :jeads )]

This is now resolved. The source of the issue is explained in bug 898492. I believe the unresponsiveness of the admin virtual machine described in that bug, was caused by processing json objects from talos that had exceptionally large quantities of replicates (up to 63,000/json object in some cases). 

I made a variety of changes to reduce the possibility of one datazilla project's object processor causing problems with another. At some point we may need to set up a completely different machine dedicated to processing talos jobs. For now, things are processing fine for b2g, no data was lost, and all b2g object processing should be caught up withing the hour.