Closed Bug 601638 Opened 14 years ago Closed 10 years ago

Proactively check test machines are ok

Categories

(Release Engineering :: General, defect, P5)

Product:
Release Engineering
Component:
General
Platform:
x86
All
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: joduinn, Unassigned)

Details

(Whiteboard: [testing][capacity]) 

Once we have test machines in production, we never recheck them for accuracy until someone reports a problem. 

How about we setup an idle-time job that runs once a month, on a weekend to have all unittest/talos machines test the same changeset, and flag any machines that report differently to the rest? There are so many intermittent tests, so hard to tell which are machine problems vs test problems. To start with, for any that are different, we can re-test those to see if its a machine problem or a test problem?

This could also automatically compare test results of new machines in staging with most recent idle-time test run, before moving the new machines to production.
John, back to you for prioritization!
Assignee: nobody → joduinn
Assignee: joduinn → nobody
OS: Mac OS X → All
Priority: -- → P5
Component: Release Engineering → Release Engineering: Automation (General)
QA Contact: release → catlee
Whiteboard: [testing][capacity]
Product: mozilla.org → Release Engineering
In our pool of test machines model, this isn't really feasible. We've got a pretty good workflow for pulling out bad machines already, too.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.