Closed Bug 1652859 Opened 4 years ago Closed 4 years ago

Disable per-commit nightly-simulation performance tests

Categories

(Testing :: mozperftest, task)

task

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: mcomella, Assigned: tarek)

Details

:sparky added per-commit tests to run on nightly-simulation builds so we could use it to try to test regressions. Unfortunately, we haven't had much success with them: we haven't been able to correlate regressions we see in the dashboards to the per-commit data. In fact, they seem to be following their own trend.

As such, I figure we should disable the tests to reduce the amount of money spent on them. We can revisit when we're further along in the process of moving FNPRMS to CI. :sparky mentioned these tests are inexpensive: if so, and they are time consuming to disable, we could consider not disabling them.

Actually, let's hold off on this: I think if we can make the per-commit tests more consistent with what we see on FNPRMS, we'll have a decent regression detection system in place. Disabling the builds will only make it more difficult to iterate on this approach.

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.