1090994 - 2.7% Linux64 A11y Regression on Inbound (v.36) Oct 28 from push 0ae1b3474b22

Status: RESOLVED WORKSFORME

(Core :: DOM: Workers, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Here is a graph showing the regression:
http://graphs.mozilla.org/graph.html#tests=%5B%5B223,131,35%5D%5D&sel=1414327504557.003,1414573005989.6677,278.2608695652174,373.9130434782609&displayrange=7&datatype=running


I did some retriggers on tbpl:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&fromchange=982fc6c2f76b&tochange=d5476a4b48eb&jobname=Ubuntu%20HW%2012.04.*%20mozilla-inbound%20talos%20other

and it was clear that this push is the pushing the values of a11yr up higher:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=0ae1b3474b22

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Catalin, Ehsan- do you have any idea if your patches could have caused this regression?

Comment 2

[Cătălin Badea (:catalinb)]

(In reply to Joel Maher (:jmaher) from comment #0)
> Here is a graph showing the regression:
> http://graphs.mozilla.org/graph.html#tests=%5B%5B223,131,
> 35%5D%5D&sel=1414327504557.003,1414573005989.6677,278.2608695652174,373.
> 9130434782609&displayrange=7&datatype=running
> 
> 
> I did some retriggers on tbpl:
> https://tbpl.mozilla.org/?tree=Mozilla-
> Inbound&fromchange=982fc6c2f76b&tochange=d5476a4b48eb&jobname=Ubuntu%20HW%201
> 2.04.*%20mozilla-inbound%20talos%20other
> 
> and it was clear that this push is the pushing the values of a11yr up higher:
> https://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?changeset=0ae1b3474b22

The patch set caused a bunch of tests to timeout - which could explain the regression. The patches have been backed out in https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=c6e5013e0b7f


I've landed an updated patch set and disabled two of the tests that were timing out:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=a1a0c1573df1
https://hg.mozilla.org/integration/mozilla-inbound/rev/746fe1b6398b

Looking at the running time graph, a1a0c1573df1 has a 1% increase over the previous changelog, but I can't say for certain if this was fixed by 746fe1b6398b.

Please let me know if you think the regression still persists.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Thanks for the reply!

c6e5013e0b7f didn't fix any regressions- odd, this might be one we have to ignore, a11y has gone up/down quite a bit lately.

looking at an alternative calculation (geometric mean instead of average):
http://graphs.mattn.ca/graph.html?geo#tests=%5B%5B223,131,35%5D%5D&sel=1414258127983.5742,1414631788298.4043,581.8965517241379,685.3448275862069&displayrange=30&datatype=running

this looks like the right changeset to cause the regression, but for some reason backing it out didn't fix it.