Last Comment Bug 476102 - auto-detect variance in talos results
: auto-detect variance in talos results
Product: Release Engineering
Classification: Other
Component: Other (show other bugs)
: other
: All All
-- normal (vote)
: ---
Assigned To: Chris AtLee [:catlee]
Depends on:
  Show dependency treegraph
Reported: 2009-01-29 21:28 PST by John O'Duinn [:joduinn] (please use "needinfo?" flag)
Modified: 2013-08-12 21:54 PDT (History)
11 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---


Description User image John O'Duinn [:joduinn] (please use "needinfo?" flag) 2009-01-29 21:28:07 PST
Until now, we rely on manual inspection to notice abnormal variance in results from talos machines. So, a human would look on graph server to see if the three machines reporting perf results for a given project branch are all within some "gutcheck" % deviation from each other.  Now, with auto-rebooting, results seem more stable, and the variance seems much reduced. 

Is there any way to check for possible variance programatically? 

Apart from being more human-friendly, this might also enable us to automatically warning people by email/tinderbox of large changes in results.
Comment 2 User image alice nodelman [:alice] [:anode] 2009-04-29 17:27:01 PDT
Moving into catlee's queue as he's working on the tool that automates this analysis.
Comment 3 User image Chris AtLee [:catlee] 2009-07-21 14:06:15 PDT
The analysis code is checked into the graph server repo at:

And is currently running via a cronjob on

Note You need to log in before you can comment on or make changes to this bug.