Open Bug 1550047 Opened 5 years ago Updated 2 years ago

Perma-fails and other backouts affect intermittent failure counts

Categories

(Testing :: General, defect, P3)

Product:
Testing
Component:
General
Version:
Version 3
Type:
defect

Tracking

(Not tracked)

People

(Reporter: gbrown, Unassigned)

References

Details

When a patch starts to cause a test failure, the resulting test failures are often initially reported against a (new or existing) intermittent failure bug. Once a pattern emerges, the patch may be backed out or a follow-up patch may resolve the issue, and later failures marked "fixed by commit", but the earlier ones remain classified against an intermittent failure bug.

This can cause confusion and cost developer time, like in bug 1549939 (new bug filed, failures investigated before realizing the failure was resolved).

It also makes our intermittent failure counts inaccurate, which can trigger triage.

I don't know how this can be avoided.

Priority: -- → P3
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.