Closed Bug 1756898 Opened 3 years ago Closed 3 years ago

create bugs for unique xpcshell failures that treeherder can use for annotating intermittent failures

Categories

(Testing :: General, task)

Product:
Testing
Component:
General
Version:
Default
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: jmaher, Unassigned)

References

(Blocks 1 open bug)

Details

When bug 1756897 is resolved, we can create a single bug for each unique test name.

This bug will:

  • create a single bug for each test case
  • use a keyword: intermittent-testcase
  • mark existing bugs for the test case as blocking this bug

:aryx, do you think the pattern of changing the keyword to intermittent-testcase would be a good idea? Also do you have thoughts on the existing bugs being blocking this bug.

Flags: needinfo?(aryx.bugmail)

another option is to remove the existing intermittent-failure keyword from existing bugs and just create new bugs with that keyword.

This is simpler in many ways. One concern I see is that if we decide to go back, we need to reverse this and that would be:

  1. removing intermittent-failure keyword from new bugs
  2. adding intermittent-failure keyword to old bugs

Reversing this would probably use the same script we used to modify the bugs in the first place.

alternatively if we go back with a new keyword, we would need to modify treeherder and the bugscache to display the old bugs (maybe there is a way to make this a toggle).

an example would be Bug 1757074. I created it manually. I am open to discussing details of summary or keywords, I think using keyword of intermittent-failure and some description in the summary is ideal.

I found this might be the simplest way forward.

The intermittent-testcase keyword should be an additional keyword assigned to the bugs in addition to intermitttent-failure. This keeps requests for all interrmittent bugs manageable, e.g. by the intermittent failures commenter bot and bug queries.

Flags: needinfo?(aryx.bugmail)

I have some code for this:
here is some code that will help with this:
https://github.com/jmaher/randomtools/tree/master/simple

85 bugs created.

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.