1243863 - win7 / mac e10s tresize regressions on aurora uplift v46

Status: RESOLVED WORKSFORME

(Testing :: Talos, defect, P4)

Description

[William Lachance (:wlach)]

tresize seems to have regressed across all platforms on the aurora uplift. We have filed bugs for Linux (bug 1242692) but I don't think we have for the others.

Regressions range from 2% for MacOSX to 10% for Windows XP

https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-aurora&originalRevision=3bfa5bc61b62&newProject=mozilla-aurora&newRevision=b9a803752a2c&framework=1&filter=tresize

Avi, what are the next steps for investigating this?

Comment 1

[Avi Halachmi (:avih)]

> tresize seems to have regressed across all platforms on the aurora uplift

Any specific patch uplift? Or just 46 riding the train from nightly, where the diff is between the last aurora 45 and the first aurora 46?

> Avi, what are the next steps for investigating this?

Hard to tell. Possibly search a similar regression while 46 was nightly which didn't get fixed. If we find one (or a series of regressions which accumulate to the change we're seeing between Au.45 and Au.46), then it's probably the likely cause for the change between aurora 45 and 46 of this test.

If we don't find one, well.. not sure, and let's hope we don't get there.

Comment 2

[William Lachance (:wlach)]

(In reply to Avi Halachmi (:avih) from comment #1)
> > tresize seems to have regressed across all platforms on the aurora uplift
> 
> Any specific patch uplift? Or just 46 riding the train from nightly, where
> the diff is between the last aurora 45 and the first aurora 46?

The latter.

> > Avi, what are the next steps for investigating this?
> 
> Hard to tell. Possibly search a similar regression while 46 was nightly
> which didn't get fixed. If we find one (or a series of regressions which
> accumulate to the change we're seeing between Au.45 and Au.46), then it's
> probably the likely cause for the change between aurora 45 and 46 of this
> test.
> 
> If we don't find one, well.. not sure, and let's hope we don't get there.

Ok, for Windows XP I pinned it to bug 1234926 (we didn't mark that as a tresize regression, but it's obvious if you look at the graph). But that leaves Win7 / MacOS X. Not seeing an obvious culprit there, looking at the graphs:

win7:

https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=[mozilla-aurora,a14119c4d02daaf55113e3945c4385f4c927da27,1]&series=[mozilla-inbound,a14119c4d02daaf55113e3945c4385f4c927da27,1]&highlightedRevisions=3bfa5bc61b62&highlightedRevisions=b9a803752a2c

mac:

https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=[mozilla-aurora,cbf4c7011d48779f12b32c6f8892db74cf84c369,1]&series=[mozilla-inbound,cbf4c7011d48779f12b32c6f8892db74cf84c369,1]&highlightedRevisions=3bfa5bc61b62&highlightedRevisions=b9a803752a2c

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

win7 falls into 2 categories:
* a bump ~jan 12 - related to bug 1240180
* bug 1243797 - where in december uplift aurora posted a lower value than inbound, now it posts the same value

I think osx just falls into the bug 1243797 category.

I am thinking of marking this as a dup of bug 1243797 - we can do more investigation if we want.

Comment 4

[Brad Lassey [:blassey] (use needinfo?)]

Kats, the assertion is that this is APZ related. If so, do you plan to fix these regression?

Comment 5

[Kartikaya Gupta (email:[e-mail])]

At least some, if not all, of the regression is likely from APZ, yes. And no, we don't have any fixes planned for the regressions. We're doing more work with APZ, and it's going to take more CPU time. We have bug 1233163 on file for trying to improve the performance but there's no magic bullet here, it will be a long series of small improvements.

Comment 6

[Milan Sreckovic [:milan] (needinfo for best results)]

Philipp, we are measuring a APZ regression with resize in performance tests.  We need some UX feedback on whether this is a noticeable problem when using Firefox and, if so, if it's a blocker for deploying APZ.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

If there is a noticeable problem, please also check if it is noticeable with e10s and no APZ (set layers.async-pan-zoom.enabled to false, restart required). e10s itself changes the resize codepath significantly so it would be good to know if the user-visible regression is from that or from the incremental slowdown from APZ.

Comment 8

[(Currently slow to respond) Philipp Sackl [:phlsa] (Firefox UX) please use needinfo]

(In reply to Milan Sreckovic [:milan] from comment #6)
> Philipp, we are measuring a APZ regression with resize in performance tests.
> We need some UX feedback on whether this is a noticeable problem when using
> Firefox and, if so, if it's a blocker for deploying APZ.

Unfortunately, this is really hard to say in a general way.
I didn't encounter an issue on my machines, but those all have some pretty high-end hardware. On lower-spec hardware and other OSes like XP, the story might be different.

I understand that this is not really a satisfying answer.
Dolske, how do we usually deal with situations like that?

Comment 9

[Justin Dolske [:Dolske]]

I'm not really sure what more I can add here, I would expect testing on low-end hardware where the problem should be at its worst (?) would be the way to start to demonstrate that APZ is a net win (which I assume is the underlying question here? tradeoff between a synthetic test and actual UX?). Otherwise someone needs to dig into the specific details of why it's a test regression, and confirm that they're known/expected tradeoffs.

I believe bsmedberg now owns the general areas of "determine if E10S performance is good enough to ship", so you might followup with him if you're looking for specific tests or metrics.

Comment 10

[George Wright (:gw280) (needinfo me!)]

Looking at this to see if it's related to bug 1179732

Comment 11

[Jim Mathies [:jimm]]

I'm not seeing a regression here - 

https://treeherder.mozilla.org/perf.html#/e10s?filter=tresize&repo=mozilla-aurora&timerange=604800