Here is a graph showing the regression: http://graphs.mozilla.org/graph.html#tests=%5B%5B223,131,35%5D%5D&sel=1414327504557.003,1414573005989.6677,278.2608695652174,373.9130434782609&displayrange=7&datatype=running I did some retriggers on tbpl: https://tbpl.mozilla.org/?tree=Mozilla-Inbound&fromchange=982fc6c2f76b&tochange=d5476a4b48eb&jobname=Ubuntu%20HW%2012.04.*%20mozilla-inbound%20talos%20other and it was clear that this push is the pushing the values of a11yr up higher: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=0ae1b3474b22
Catalin, Ehsan- do you have any idea if your patches could have caused this regression?
(In reply to Joel Maher (:jmaher) from comment #0) > Here is a graph showing the regression: > http://graphs.mozilla.org/graph.html#tests=%5B%5B223,131, > 35%5D%5D&sel=1414327504557.003,1414573005989.6677,278.2608695652174,373. > 9130434782609&displayrange=7&datatype=running > > > I did some retriggers on tbpl: > https://tbpl.mozilla.org/?tree=Mozilla- > Inbound&fromchange=982fc6c2f76b&tochange=d5476a4b48eb&jobname=Ubuntu%20HW%201 > 2.04.*%20mozilla-inbound%20talos%20other > > and it was clear that this push is the pushing the values of a11yr up higher: > https://hg.mozilla.org/integration/mozilla-inbound/ > pushloghtml?changeset=0ae1b3474b22 The patch set caused a bunch of tests to timeout - which could explain the regression. The patches have been backed out in https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=c6e5013e0b7f I've landed an updated patch set and disabled two of the tests that were timing out: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=a1a0c1573df1 https://hg.mozilla.org/integration/mozilla-inbound/rev/746fe1b6398b Looking at the running time graph, a1a0c1573df1 has a 1% increase over the previous changelog, but I can't say for certain if this was fixed by 746fe1b6398b. Please let me know if you think the regression still persists.
Thanks for the reply! c6e5013e0b7f didn't fix any regressions- odd, this might be one we have to ignore, a11y has gone up/down quite a bit lately. looking at an alternative calculation (geometric mean instead of average): http://graphs.mattn.ca/graph.html?geo#tests=%5B%5B223,131,35%5D%5D&sel=1414258127983.5742,1414631788298.4043,581.8965517241379,685.3448275862069&displayrange=30&datatype=running this looks like the right changeset to cause the regression, but for some reason backing it out didn't fix it.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.