Range: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=dfcfa00eb188&tochange=390072430c56 Given that all of Vivien's stuff only touches mobile/, it is not a factor here.
I have this back out locally, but I'm rebuilding to make sure I don't make the tree go up in flames. This will also back out https://hg.mozilla.org/mozilla-central/rev/437f175609b8 (bug 654990) because it depends on stuff in this range.
Back out: http://hg.mozilla.org/mozilla-central/rev/dd9ba28d2bd9 Merge (backs out bug 654990): http://hg.mozilla.org/mozilla-central/rev/65316725d03b Leaving the tree closed until we confirm that this was the regression (although it likely was).
I suspect this is related to infrastructure problems or other non-code changes. In the OSX-tp4 graph, the last good test is on dfcfa00eb188 and the first bad test is on b2ce3817d034, even though only the "mobile" directory changed between them. In the Win7-tp4 graph, the last good test is on dfcfa00eb188 and the first bad test is on 9e31df64bfd7, which was an *earlier* changeset whose Tp4 test happened to run *later*.
Yup, this was due to a scheduled change to Talos: https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/Gu7oVmAf764
One changeset was missed in the backout of bug 514437; I backed it out here to fix oranges: http://hg.mozilla.org/mozilla-central/rev/e0f6db50231f As we'd expect based on comment 3 and comment 4, the backout does not seem to have fixed the regression.
(In reply to comment #4) > Yup, this was due to a scheduled change to Talos: > https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/Gu7oVmAf764 We should hold off on reopening the tree until we get a confirmation on that. I'm also inclined to say that we should back out everything that landed today to get a baseline for new numbers if all that did in fact land. With all that being said, that *only* explains the tp4 regression and not the Ts and TSVG regressions.
It explains the TSVG regression at least; I would fully expect bug 651659 to affect TSVG. I really hope it doesn't affect Ts, though....
We don't need to back things out, but we do need to trigger a bunch of talos runs.
(which Matt is doing a great job of right now!)
Looking at the first results coming in from re-triggered Talos runs on older changesets, it looks like they are showing the exact same regressions as the later changesets. (If this holds true, it means the regression is entirely due to the Talos updates.)
On Places branch I see similar regressions on a changeset that only fixed some cpp warnings. Checking with compare-talos on central looks like all changesets between vivien's and current tip did not cause any regression from the new baseline. The tip has double talos thanks to nightlies too. I suspect the tree can be reopened.
I reopened the tree based on Matt and Marco's assessment.
(In reply to comment #12) > I reopened the tree based on Matt and Marco's assessment. ...which means this bug can be closed :)