Closed
Bug 457780
Opened 16 years ago
Closed 10 years ago
Write tests for regression tracking per-machine
Categories
(Webtools Graveyard :: Graph Server, defect)
Webtools Graveyard
Graph Server
Tracking
(Not tracked)
RESOLVED
WONTFIX
1.0
People
(Reporter: rdoherty, Assigned: lars)
Details
Sometimes a machine will report back test results that are way off from the other 2. We should have tests that run periodically and verify the test results are no more than a certain % difference.
Reporter | ||
Updated•16 years ago
|
Assignee: nobody → mark
Assignee | ||
Updated•16 years ago
|
Assignee: mark → lars
Assignee | ||
Comment 1•15 years ago
|
||
before any serious work can be done on this, there are some questions that need to be answered: 1 - how does this application run? cron? invoked by a Web app? 2 - how does this app get configured? how does one specify the "certain % difference" ? 3 - in what form does this app report its findings? return data via stdout? generates a web page? generates json or xml data? updates stats in a database?
Reporter | ||
Comment 2•15 years ago
|
||
Initial specs: 1) Daily via cron - benchmarking needed to determine if we can decrease this interval 2) Test for more than 1 standard deviation from the previous 3 days of data from the latest test run Flag machine for problems, manually verify fixed or Also need to flag tests 3) HTML page that shows all machines as red or green List failed tests under machines We will need some way of un-flagging a machine or test one the issue has been resolved. Command line tool might be easiest, avoids the need for user accounts, etc. We will need some way to track what machines and tests were flagged as failed. A table or two in the database will most likely be necessary.
Updated•15 years ago
|
Summary: Write tests for machine agreement → Write tests for regression tracking per-machine
Assignee | ||
Comment 3•10 years ago
|
||
old, outdated
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WONTFIX
Updated•8 years ago
|
Product: Webtools → Webtools Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•