Mysterious 25% CART improvement on beta (v30) seen on win8 and landed on April 30

RESOLVED INVALID

Status

()

defect
RESOLVED INVALID
5 years ago
5 years ago

People

(Reporter: jmaher, Assigned: mikedeboer)

Tracking

({perf, regression})

Trunk
x86
Windows 8
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [talos_regression])

the cart likes to roll downhill, and in this case a patch landed directly on beta is causing a big regression.

Here is the graph:
http://graphs.mozilla.org/graph.html#tests=[[309,53,31]]&sel=none&displayrange=30&datatype=running

there is no alert email- my neural nets are getting faster than the alert emails!

this is the pushlog:
https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?changeset=23481585e2e5

more information about cart is here:
https://wiki.mozilla.org/Buildbot/Talos/Tests#TART.2FCART
please look into this asap as this is on beta and the longer we wait, the greater chance of shipping with this.
ooooh, I think this is caused by bug 966723. When does the cart test start measuring the animation duration?

Before bug 966723, the animation the exit Customize Mode start on mousedown - iow as soon as the left mouse button hits the menu-button.
After bug 966723, it starts when after a full click - iow as soon as the left mouse button is released.

So, to paraphrase my question, how does the cart test synthesize the mouse events to enter/ leave Customization Mode and does it use the menu-button as the target?
Flags: needinfo?(jmaher)
Flags: needinfo?(avihpit)
Assignee: nobody → mdeboer
Oh, and Joel, your neural nets _are_ impressive ;)
Hm - CART bypasses the UI and just calls gCustomizeMode.enter/.exit in order to trigger the transitions... so, no mouse events, from what I can tell... tricky.
Flags: needinfo?(jmaher)
Flags: needinfo?(avihpit)
(In reply to Mike Conley (:mconley) from comment #4)
> Hm - CART bypasses the UI and just calls gCustomizeMode.enter/.exit in order
> to trigger the transitions... so, no mouse events, from what I can tell...

Right.

- The 1-customize-enter.* results are measured between calling gCustomizeMode.enter() until the customizationready event is handled.

- The 2-customize-exit.* results are measured between calling gCustomizeMode.exit() until the aftercustomization event is handled.

- The 3-customize-enter-css.* results are measured between calling gCustomizeMode.enter() until the customization-transitionend event is handled.
Well, in that case I'll be doing some try pushes to see which of the two changesets caused the regression.
Also, how can I spot the regression in the chart that Joel linked?
nvmd, kinda found the datapoints.
Sooooooo - looking at this graph, we're seeing CART actually go _down_. That's a _good_ thing. By all accounts, we're seeing dramatic CART improvements here.

\o/?

The question is "why"?

Here are the Datazilla graphs for that time period for win8 on mozilla-beta:

https://datazilla.mozilla.org/?start=1397580540&stop=1398955348&product=Firefox&repository=Mozilla-Beta&os=win&os_version=6.2.9200&test=cart&graph_search=c95ac01e332e&tr_id=5303340&graph=1-customize-enter.all.TART&project=talos
Summary: 25% CART regression on beta (v30) seen on win8 and landed on April 30 → Mysterious 25% CART improvement on beta (v30) seen on win8 and landed on April 30
This is an improvement which got filed accidentally as a regression.

If everyone's fine with it, invalid ;)
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.