Closed Bug 1296643 Opened 8 years ago Closed 5 years ago

[OrangeFactor v2] Reconcile difference between TestFailures and OrangeFactor

Categories

(Testing Graveyard :: ActiveData, defect)

Product:
Testing Graveyard
Component:
ActiveData
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: ekyle, Assigned: ekyle)

References

Details

The TestFailures dashboard sees the test-level intermittent failures.  This is not all failures, there are job-level failures that can be seen in TH that can not be seen in TestFailures.  ActiveData has the job level failures, and they plan to be shown using the TestFailures dashboard also.  The test-level failures will be subtracted from the job-level failures to leave a residue which needs to be explored before OFv2 can replace OF.

To do this, bug and star information from TH is required.  Finding the pattern in the residue will involve testing set operations (queries) over both datasets to establish the heuristics that explain the difference between TestFailures and Treeherder/OrangeFactor.   This will be a continuous process over several months, so recent data is best.
Depends on: 1296634
No longer depends on: 1296634
Depends on: 1296634
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.