Disable per-commit nightly-simulation performance tests
Categories
(Testing :: mozperftest, task)
Tracking
(Not tracked)
People
(Reporter: mcomella, Assigned: tarek)
Details
:sparky added per-commit tests to run on nightly-simulation
builds so we could use it to try to test regressions. Unfortunately, we haven't had much success with them: we haven't been able to correlate regressions we see in the dashboards to the per-commit data. In fact, they seem to be following their own trend.
As such, I figure we should disable the tests to reduce the amount of money spent on them. We can revisit when we're further along in the process of moving FNPRMS to CI. :sparky mentioned these tests are inexpensive: if so, and they are time consuming to disable, we could consider not disabling them.
Reporter | ||
Comment 1•4 years ago
|
||
Actually, let's hold off on this: I think if we can make the per-commit tests more consistent with what we see on FNPRMS, we'll have a decent regression detection system in place. Disabling the builds will only make it more difficult to iterate on this approach.
Description
•