39% SVGX OSX 10.8 regression on inbound (v.37) Dec 31 from push db6bdc09068d

RESOLVED FIXED

Status

()

Core
Graphics
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: jmaher, Unassigned)

Tracking

({perf, regression})

Trunk
x86_64
Mac OS X
perf, regression
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [talos_regression], gfx-noted)

we are in the process of switching over to use geometric mean instead of average (where we drop the highest value and average the remaining).  This will affect some tests more uniquely than others, in this case we have a svgx regression:
https://groups.google.com/forum/#!forum/mozilla.dev.tree-alerts

this regression is not much of an issue with the averages (although it is slightly up).  Here is a graph to show the issue:
http://graphs.mozilla.org/graph.html#tests=%5B%5B281,63,24%5D%5D&sel=1419556573716.223,1420455795014.0598,-4.547473508864641e-13,737.7049180327867&displayrange=30&datatype=geo

You can see that the geometric mean (drop down on the top) shows a significant regression, but switching to average yields a minimal regression.

I did some retriggers on treeherder:
https://treeherder.mozilla.org/ui/#/jobs?repo=mozilla-inbound&filter-searchStr=Rev5%20MacOSX%20Mountain%20Lion%2010.8%20mozilla-inbound%20talos%20svgr&fromchange=f996eb4935c6&tochange=f7dbc64435bc

You can see the regression happens from this push:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=be2eca32172a

Updated

4 years ago
Flags: needinfo?(bgirard)

Comment 1

4 years ago
(In reply to Joel Maher (:jmaher) from comment #0)
> You can see that the geometric mean (drop down on the top) shows a
> significant regression, but switching to average yields a minimal regression.

Unless other evidence show up (for instance, being able to see individual sub-results over time, e.g. if datazilla was working), I'd attribute it to better coverage and accuracy of the geometric mean than to an issue with it.
Whiteboard: [talos_regression] → [talos_regression], gfx-noted
I did some testing and the display list dumping isn't accidentally turned on by the looks for it. I'm not sure.
I did a few more retriggers and I'm also seeing some 400-410 scores on the previous revision. Are you sure it's mine? Not looking to shift the blame but my one patch causes us to do less work and the otherwise is behind a branch that doesn't seem to be accidentally triggered. It just causes a tiny bit of code bloat.

https://hg.mozilla.org/integration/mozilla-inbound/rev/db6bdc09068d
Flags: needinfo?(bgirard) → needinfo?(jmaher)
:BenWa, yes- it appears I had miscategorized this.  My apologies, it does look to have started on the previous push.

Terrence, can you weigh in on why your changes from bug 1110931 appears to have cause SVGX on OSX 10.8 to increase in value?
Blocks: 1110931
No longer blocks: 1113781, 1113837
Flags: needinfo?(jmaher) → needinfo?(terrence)
Summary: 39% SVGX OSX 10.8 regression on inbound (v.37) Dec 31 from push be2eca32172a → 39% SVGX OSX 10.8 regression on inbound (v.37) Dec 31 from push db6bdc09068d
Whiteboard: [talos_regression], gfx-noted → [talos_regression]
Yes, probably, this patch makes access to heapState happen in a threadsafe manner. I'll back it out and try to figure out if a weaker ordering will still be racy or not.
Flags: needinfo?(terrence)
Thanks Terrence!
Whiteboard: [talos_regression] → [talos_regression], gfx-noted
Joel, is the regression fixed?
I see a drop (hard without more data points) to fix about half this regression.

We were ~280, then regressed up to ~325, now we are ~300.  With a few more data points we could know with more certainty.
this looks a lot better with a few more data points:
http://graphs.mozilla.org/graph.html#tests=%5B%5B281,63,24%5D%5D&sel=1419633023302.4473,1420916860633,0,774.1935483870966&displayrange=30&datatype=geo
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.