2007002 - [meta] Implement an automated smart backfill system for performance tests

Status: NEW

(Tree Management :: Perfherder, task, P1)

Description

[Greg Mierzwinski [:sparky]]

This bug is for implementing an automated smart backfill system. It should mostly live in the existing backfill bot (Sherlock), and will need to have the change detection techniques being used integrated into it so that it can determine how to do a binary search backfill for the culprit.

There are two main parts to this task:
(1) Integrate the change detection technique into the backfill bot.
(2) Change the backfill bot to use those techniques when deciding where to schedule the tests to run.

Comment 1

[Greg Mierzwinski [:sparky]]

In bug 1693531, we started discussions around having a binary backfill method however it was a basic variant that does not rely on the data that we produce. Instead it would be triggered by sheriffs. Here, we're specifically looking for something automated that does not need to be triggered by sheriffs. Ideally, these backfills are complete by the time a sheriff begins triaging an alert so that they can immediately file a bug after some verification.