Closed Bug 461012 Opened 13 years ago Closed 13 years ago

Try-talos missed perf regression in patch, which mozilla-central-talos did catch

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: joduinn, Unassigned)

Details

In bug#449443, sdwilsh submitted a patch to try-talos, and the results looked fine. However, when he landed the same patch on mozilla-central, the mozilla-central-talos results showed a perf regression. 

Its hard to recreate this problem, but what can we do on try-talos to catch this kinda problem in future? 

Filing this bug to track this, as irc/email is too easy to lose...
When the try-talos run was done, what were the results compared to ?
I had done a clean build of m-c at the time, and I also looked at past runs with patches that didn't affect perf.
From triage today, there are also some other issues to look at:

- no comparison numbers per run / no baseline numbers
- skips queued builds, which is being tracked in bug#457885
- long time since last reboot of slaves, which also causes variance
- only have one slave per platform, so cant cross-compare trio-of-results as in m-c talos.

Pushing to p3/future.
Component: Release Engineering → Release Engineering: Future
try talos:

- no longer skips queued builds
- reboots after every test run
- has multiple slaves per platform and runs each test twice

Going to call this fixed from the list in comment #3.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Moving closed Future bugs into Release Engineering in preparation for removing the Future component.
Component: Release Engineering: Future → Release Engineering
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.