1004460 - Mysterious 25% CART improvement on beta (v30) seen on win8 and landed on April 30

Status: RESOLVED INVALID

(Firefox :: Toolbars and Customization, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

the cart likes to roll downhill, and in this case a patch landed directly on beta is causing a big regression.

Here is the graph:
http://graphs.mozilla.org/graph.html#tests=[[309,53,31]]&sel=none&displayrange=30&datatype=running

there is no alert email- my neural nets are getting faster than the alert emails!

this is the pushlog:
https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?changeset=23481585e2e5

more information about cart is here:
https://wiki.mozilla.org/Buildbot/Talos/Tests#TART.2FCART

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

please look into this asap as this is on beta and the longer we wait, the greater chance of shipping with this.

Comment 2

[Mike de Boer [:mikedeboer]]

ooooh, I think this is caused by bug 966723. When does the cart test start measuring the animation duration?

Before bug 966723, the animation the exit Customize Mode start on mousedown - iow as soon as the left mouse button hits the menu-button.
After bug 966723, it starts when after a full click - iow as soon as the left mouse button is released.

So, to paraphrase my question, how does the cart test synthesize the mouse events to enter/ leave Customization Mode and does it use the menu-button as the target?

Comment 3

[Mike de Boer [:mikedeboer]]

Oh, and Joel, your neural nets _are_ impressive ;)

Comment 4

[Mike Conley (:mconley) (:⚙️)]

Hm - CART bypasses the UI and just calls gCustomizeMode.enter/.exit in order to trigger the transitions... so, no mouse events, from what I can tell... tricky.

Comment 5

[Avi Halachmi (:avih)]

(In reply to Mike Conley (:mconley) from comment #4)
> Hm - CART bypasses the UI and just calls gCustomizeMode.enter/.exit in order
> to trigger the transitions... so, no mouse events, from what I can tell...

Right.

- The 1-customize-enter.* results are measured between calling gCustomizeMode.enter() until the customizationready event is handled.

- The 2-customize-exit.* results are measured between calling gCustomizeMode.exit() until the aftercustomization event is handled.

- The 3-customize-enter-css.* results are measured between calling gCustomizeMode.enter() until the customization-transitionend event is handled.

Comment 6

[Mike de Boer [:mikedeboer]]

Well, in that case I'll be doing some try pushes to see which of the two changesets caused the regression.

Comment 7

[Mike de Boer [:mikedeboer]]

Also, how can I spot the regression in the chart that Joel linked?

Comment 8

[Mike de Boer [:mikedeboer]]

nvmd, kinda found the datapoints.

Comment 9

[Mike Conley (:mconley) (:⚙️)]

Sooooooo - looking at this graph, we're seeing CART actually go _down_. That's a _good_ thing. By all accounts, we're seeing dramatic CART improvements here.

\o/?

The question is "why"?

Here are the Datazilla graphs for that time period for win8 on mozilla-beta:

https://datazilla.mozilla.org/?start=1397580540&stop=1398955348&product=Firefox&repository=Mozilla-Beta&os=win&os_version=6.2.9200&test=cart&graph_search=c95ac01e332e&tr_id=5303340&graph=1-customize-enter.all.TART&project=talos

Comment 10

[Avi Halachmi (:avih)]

This is an improvement which got filed accidentally as a regression.

If everyone's fine with it, invalid ;)