1015569 - 63% a11y regression on Win7/Win8 PGO may 21st on fx-team (fx32) from push 84c572a9c878

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

This is the first a11y regression that we have found!  Congrats to all of you on this bug- you have found a first :)

It is rare that we find PGO only regressions, but if you think about it they are the most important because that is what we actually ship.

Here is a graph showing the problem:
http://graphs.mozilla.org/graph.html#tests=[[223,64,25]]&sel=1400643559000,1400816359000&displayrange=7&datatype=running

Here is the changeset:
http://hg.mozilla.org/integration/fx-team/pushloghtml?changeset=84c572a9c878

I had to do some builds (we don't PGO build everything) and then did some retriggers in case there was some noise, this is solid:
https://tbpl.mozilla.org/?tree=Fx-Team&fromchange=b9c220c000a5&tochange=2a59bd590870&jobname=Windows%207%2032-bit%20fx-team%20pgo%20talos%20other

Here is some information about the tests:
https://wiki.mozilla.org/Buildbot/Talos/Tests#a11y

If you are patient, datazilla will load and show you the problem is in tablemutation.html:
https://datazilla.mozilla.org/?start=1400173656&stop=1400873582&product=Firefox&repository=Fx-Team&os=win&os_version=6.1.7601&test=a11yr&graph_search=84c572a9c878&x86_64=false&project=talos

Comment 1

[Avi Halachmi (:avih)]

(In reply to Joel Maher (:jmaher) from comment #0)
> It is rare that we find PGO only regressions, but if you think about it they
> are the most important because that is what we actually ship.

True, however, it does bring up an important question of why only on PGO?

Also, because this test has never shown regressions before, it puts a question mark both over it's effectiveness and over the validity of this regression.

Since the test description is rather sparse, we'll probably have to analyze and assess the test as well.

Of course, this shouldn't prevent also investigating the offending patch independently.

Comment 2

[Avi Halachmi (:avih)]

ni myself as a reminder to look into this.

Comment 3

[Tim Taubert [:ttaubert] (inactive)]

Looks like the graph went down by a lot. Another regression solved by bug 1016942?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

the graphs are down, I vote for calling this fixed!

Comment 5

[David Bolter [:davidb] (NeedInfo me for attention)]

Seconded :)

Comment 6

[Avi Halachmi (:avih)]

Good. I'll keep the ni?self to still examine this test at some stage.

Comment 7

[Avi Halachmi (:avih)]

Oops :p

Comment 8

[Avi Halachmi (:avih)]

Fix the reminders - remove the needinfo ;)