1272716 - Perfherder - subtest alerts should use historical vs forward comparisons instead of revision comparisons

Status: RESOLVED INCOMPLETE

(Tree Management :: Perfherder, task, P3)

Description

[Bob Clary [:bc] (inactive)]

1. https://treeherder.mozilla.org/perf.html#/alerts?status=0&framework=3

2. click subtests

   https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=fx-team&originalRevision=c46493e9c26bc7b16ce052af4c023b239e4ce5ec&newProject=fx-team&newRevision=a0aa4227b6b36846a06ad580dd30628316cf18f7&originalSignature=b6afe5e3241fe6db49c04fc60855d1a8259d435f&newSignature=b6afe5e3241fe6db49c04fc60855d1a8259d435f&framework=3

The subtests are doing a comparison between the one old and one new revisions. jmaher suggested this should be doing the historical vs. forward comparison statistics instead.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

when viewing an alert, it is deceptive to see suspect revision vs previous revision where each has 1 data point- I would think we would show the information for the subtests using the fore_window, back_window.

Comment 2

[William Lachance (:wlach)]

I agree this might be better, though it entails designing an entirely new comparison view with appropriate UI.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

yeah, not trivial to solve- possibly some wording,etc. could help until we have a plan for solving this more generically- I think the e10s dashboard work would be a good starting point.

Comment 4

[William Lachance (:wlach)]

Bug 1273513 adds yet another type of view which uses this window (in this case to handle base comparisons for manually generated compares). It's not quite the ask here, but it's close.

Comment 5

[Ionuț Goldan [:igoldan]]

This ticket is much older than 18 months, thus closing it as INCOMPLETE.