1056219 - 10% tsvgx osx 10.8 regression on future push to inbound (v.34) from bug 1053517

Status: RESOLVED FIXED

(Core :: Networking: Cache, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

Looking at the try push:
https://tbpl.mozilla.org/?tree=Try&rev=9f5b1cf9e99f

I compared the numbers to m-c and see that osx 10.8 tsvgx is about 10% higher.

The range is normally:
548.5 -> 652.0

the value I get is 703.9.

This is a future bug from a conversation on IRC, so keep an eye out for it when bug 1053517 is landed.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

here is the email I got:
Regression: Mozilla-Inbound - SVG-ASAP - MacOSX 10.6 (rev4) - 15.7% increase
----------------------------------------------------------------------------
    Previous: avg 610.045 stddev 22.380 of 12 runs up to revision ebffbc56b3a7
    New     : avg 706.026 stddev 1.045 of 12 runs since revision 25c918c5f3e1
    Change  : +95.981 (15.7% / z=4.289)
    Graph   : http://mzl.la/1uYPsST

I assume 10.8 hasn't ran yet (it is backlogged).

Comment 2

[Honza Bambas (:mayhemer)]

I've spent enormous amount of time to make sure that specifically tsvgx is unaffected by cache2.  

Can you please provide two try csets I could compare on http://compare-talos.mattn.ca/ ?  (If you don't have I'll run them myself).  I would be interested which exact sub-test is the one being slow.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Honza, you have really fixed a lot of these issues- here is a link to datazilla to show the exact difference:
https://datazilla.mozilla.org/?start=1408027561&stop=1408632361&product=Firefox&repository=Mozilla-Inbound&os=mac&os_version=OS%20X%2010.6.8&test=tsvgx&graph_search=87553a321db8&x86=false&project=talos

the pages that are affected are:
composite-scale-opacity.svg
composite-scale-rotate-opacity.svg
composite-scale-rotate.svg
composite-scale.svg

the rest is not of concern.

Comment 4

[Honza Bambas (:mayhemer)]

Joel, thanks!  These are new to me.  That is the price of leaving the talos on cache1 so long time :((

I assume these tests work similarly as others - the svg file is loaded some 20 - 30 times and the overall time is reported, is that so?

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

yes, we run each page 25 times and we ignore the first 5 and report a median of the remaining 20 as a single value for each page to graph server.  For datazilla it shows the median of all the values, and you can click on a single data point to get the raw values.

Comment 6

[Lawrence Mandel [:lmandel] (use needinfo)]

Joel - Is this still an issue?
Honza - If this is still valid, can you investigate as 34 moves to Beta on Monday?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

this appears to be active in v.34:
http://graphs.mozilla.org/graph.html#tests=%5B%5B281,63,21%5D,%5B281,52,21%5D%5D&sel=1405247018348,1413023018348&displayrange=90&datatype=running

It was fixed Sept 3 on v.35, so maybe there is something we can do.  Unfortunately the tbpl data is deleted after 30 days, so I can't go in and verify if all tests were run, but my best guess is we fixed it with:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=4effa50d8934

I don't see a data point for that on the graph, but for the push after it (a test only fix) it is fixed and 2 pushes prior it is still regressed.

Comment 8

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Lawrence Mandel [:lmandel] from comment #6)
> Joel - Is this still an issue?
> Honza - If this is still valid, can you investigate as 34 moves to Beta on
> Monday?
I guess you wanted to ask "[e-mail]" for the feedback.
Honza

Comment 9

[Jason Duell]

Honza is out on PTO through at least the end of this week.

If I'm interpreting comment 7 correctly, it looks like this OSX-only perf regression has been fixed by something in the changelog range that Joel posted.  And we might want to uplift that code if we're worried about the result here.  But we don't know exactly which changeset(s) are the good ones.

This smells to me like something that could just ride the train?  A 10% regression on one Talos test on one platform for one release isn't going to kill us.  OTOH there's only 3 bugs in that changeset, and only bug 982338 looks like it has a lot of dependencies, so maybe uplifting is worth it.  I'll leave that call to Lawrence.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

I vote for riding the trains; trunk and aurora look good, we just have a slight tick in beta which will be fixed in the next revision:
http://graphs.mozilla.org/graph.html#tests=%5B%5B281,52,21%5D,%5B281,63,21%5D,%5B281,53,21%5D%5D&sel=1411321590129,1413913590129&displayrange=30&datatype=running

Comment 11

[Lawrence Mandel [:lmandel] (use needinfo)]

Thanks for the feedback. Based on comment 9 and comment 10, I'm going to resolve this bug as fixed in 35 and wontfix for 34.