Perfherder - subtest alerts should use historical vs forward comparisons instead of revision comparisons

NEW
Unassigned

Status

Tree Management
Perfherder
2 years ago
5 months ago

People

(Reporter: bc, Unassigned)

Tracking

Details

(Reporter)

Description

2 years ago
1. https://treeherder.mozilla.org/perf.html#/alerts?status=0&framework=3

2. click subtests

   https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=fx-team&originalRevision=c46493e9c26bc7b16ce052af4c023b239e4ce5ec&newProject=fx-team&newRevision=a0aa4227b6b36846a06ad580dd30628316cf18f7&originalSignature=b6afe5e3241fe6db49c04fc60855d1a8259d435f&newSignature=b6afe5e3241fe6db49c04fc60855d1a8259d435f&framework=3

The subtests are doing a comparison between the one old and one new revisions. jmaher suggested this should be doing the historical vs. forward comparison statistics instead.
when viewing an alert, it is deceptive to see suspect revision vs previous revision where each has 1 data point- I would think we would show the information for the subtests using the fore_window, back_window.
I agree this might be better, though it entails designing an entirely new comparison view with appropriate UI.
yeah, not trivial to solve- possibly some wording,etc. could help until we have a plan for solving this more generically- I think the e10s dashboard work would be a good starting point.
Bug 1273513 adds yet another type of view which uses this window (in this case to handle base comparisons for manually generated compares). It's not quite the ask here, but it's close.
You need to log in before you can comment on or make changes to this bug.