987875 - CART regression on win8 seen on fx-team

Status: RESOLVED WONTFIX

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

found a talos alert here:
https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/cM8vJrtanAU

looking at datazilla you can see the regression in the 1-* tests:
https://datazilla.mozilla.org/?start=1395164852&stop=1395769652&product=Firefox&repository=Fx-Team&os=win&os_version=6.2.9200&test=cart&graph_search=f2c21183e206,c148f0b0c8b,&tr_id=4909851&graph=1-customize-enter.error.TART&project=talos

it looks like we bump up a bit on a merge:
https://tbpl.mozilla.org/?tree=Fx-Team&rev=c148f0b0c8b4

and then again the next changeset:
https://tbpl.mozilla.org/?tree=Fx-Team&rev=f2c21183e206

this is where the talos alert picks up the change in the second changeset. The talos alert is for pgo where as I am showing non pgo on the graph.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

want to get this flagged.  I assume this slipped through the cracks as it is a few weeks old now.

Comment 2

[Benjamin Kerensa [:bkerensa]]

Can you explain impact so we can make a decision on tracking?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

this is 3.72% windows 8 customized animation test- maybe Avi can explain what the end user will experience when the cart test regresses

Comment 4

[Avi Halachmi (:avih)]

This tests the smoothness/performance of the "customize" animation (australis -> click the burger menu -> customize).

Also, since we can't really quantify "smoothness" into a single number, the test result which regressed here is an average of quite a few sub tests (smoothness, speed, in entering or exiting customize, extra lag before the animation starts, etc).

With 3.7% regression - probably no one would notice anything in practice.

Though if we had lots of small regressions like this, eventually it will be noticeable.

Comment 5

[Lukas Blakk [:lsblakk] use ?needinfo]

Unless this lands to central and has similar impact, we cannot track issues on project branches.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

this is 6+ months old now, I am going to resolved/wontfix- please reopen if you disagree.