1825127 - Run mann-whitney-u change detector on autoland at tier-3

Status: NEW

(Testing :: Raptor, task, P2)

Description

[Greg Mierzwinski [:sparky]]

This bug is for running the mann-whitney-u change detector mwu on autoland at tier-3.

We'll need to determine the settings that the mwu detector should run with in this case. The existing mwu task only looks at two revisions, and this new task will need to look at a range of commits.

How large should the range of commits be? There's a small "expanding-window" in our current technique that would limit a maximum data (we would need a way to change it): https://github.com/mozilla/mozperftest-tools/blob/823f58ee1530fb4c488338fbbfe9acd1ce9b4aad/mozperftest_tools/mozperftest_tools/regression_detector.py#L364

If we set the window to 1, then we'll only incorporate one extra push on each side of the comparison. This would mean that if we detect a change in one push, we only look back 4 pushes, and we detect the change in the current push, then we would continuously detect a change there for the next 3-4 runs.

We'll need to disallow alerting when a change is detected on the current revision (or an edge revision) so that we can take advantage of the possibility of the window increasing the detection precision if we have a borderline detection.