Detecting regressions came up as a top priority in the Fennec 1.0 post-mortem. These have been going to me directly so I could help prevent spamming anyone with potentially confusing information (if they were too noisy to be useful, for example), but these seem like they're not too spammy so they should go somewhere public. I'd say moz.dev.tree-management as well, so Firefox and Fennec developers can see these. They currently look like this, though with links: From: Nobody <email@example.com> Date: Sat, 30 Jan 2010 14:17:05 -0000 To: firstname.lastname@example.org Subject: Talos Regression: Tp4 increase 22.19% on Nokia n810 mobile-tracemonkey Regression: Tp4 increase 22.19% on Nokia n810 mobile-tracemonkey Previous results: 27156.3 from build 20100128022652 of revision 269af0364214 at 2010-01-28 11:40:10 on maemo-n810-53 run # 0 New results: 33182.3 from build 20100128022652 of revision 269af0364214 at 2010-01-28 21:40:14 on maemo-n810-41 run # 0 Suspected checkin range: from revision 269af0364214 to revision 269af0364214
cc'ing email@example.com. Also, I marked bug 545472 as a dependency, as the revision linked to and specified is the mobile-browser revision, but the link goes to the tracemonkey repo. I'm guessing that the mozilla-* version is going to be more granular and potentially more useful to track than the mobile-browser version (we can currently only track one in graphs; extending that is significantly beyond the scope of this bug).
Also, the email above appears to be the same buildid ? Certainly possible there's device-specific noise here.
The Tp4 graph is questionably useful, as most runs crash mid-run and the resulting graph is very very noisy. http://3.ly/1Xwo The Tdhtml graph, however, looks like a valid regression. http://3.ly/i2v1 I think I should fix the revision (mobile-browser -> mozilla-* revision, bug 545472) then ask catlee to send these to moz.dev.tree-management.
Ok, I think bug 545472 might take longer; in the meantime I've gotten two more regression emails (DHTML increase 11.24% on m-c, 12.03% on tm). Catlee: could we send these emails to moz.dev.tree-management along with the Firefox desktop talos emails? Assuming you have no objections.