798130 - Regression in startup in 18.

Status: RESOLVED INVALID

(Firefox :: General, defect)

Description

[(dormant account)]

The long tail is pretty bad in ff18 compared to previous release http://is.gd/S7x4yv

Comment 1

[(dormant account)]

According to evolution looks like things started trending up on sep11th. 
http://is.gd/1t5pIv

Comment 2

[Nathan Froyd [:froydnj]]

That would probably be the IonMonkey merge, though dev.tree-management doesn't appear to have any Ts regressions.  There are some ~5% Tp5 regressions around that time or shortly thereafter, though.

Comment 3

[(dormant account)]

(In reply to Nathan Froyd (:froydnj) from comment #2)
> That would probably be the IonMonkey merge, though dev.tree-management
> doesn't appear to have any Ts regressions.  There are some ~5% Tp5
> regressions around that time or shortly thereafter, though.

It lines up with ionmonkey, but this regression is especially obvious on the tail and I can't imagine something cpu-bound could affecting startups >30s.

Comment 4

[David Anderson [:dvander] - inactive, e-mail if emergency]

I don't appear to have access to that site so I can't see the data. It would indeed be strange if it was IonMonkey - we have limits in place to curb the compilation process from exploding and haven't seen evidence of that in the wild yet. I guess it increased the binary size by some non-trivial amount, I think around 1MB?

Comment 5

[Matthew N. [:MattN]]

(In reply to David Anderson [:dvander] from comment #4)
> I don't appear to have access to that site so I can't see the data.

Login with Persona using any email address.

Comment 6

[brendan c]

Attachment: screen cap of SIMPLE_MEASURES_FIRSTPAINT histogram

Comment 7

[brendan c]

Attachment: screen cap of SIMPLE_MEASURES_FIRSTPAINT evolution

from http://is.gd/1t5pIv

Comment 8

[brendan c]

Taras, I want to make sure we are on the same page (literally looking at the same telemetry page)--

When I follow your first link ( http://is.gd/S7x4yv ) the highest startup times (>30s) are better in 18 than 17. (see screen cap https://bugzilla.mozilla.org/attachment.cgi?id=668621 )

And looking at http://is.gd/1t5pIv (see https://bugzilla.mozilla.org/attachment.cgi?id=668625 ) Sept. 11 doesn't look especially interesting (could just be noise); in fact, the series seems to dip locally after that day...

Please let me know if ended up looking at the wrong graphs, or if I'm missing the feature you are concerned about, interpreting something differently, etc..

Comment 9

[(dormant account)]

(In reply to Taras Glek (:taras) from comment #1)
> According to evolution looks like things started trending up on sep11th. 
> http://is.gd/1t5pIv

Sep 11 regression is also confirmed by our io tracker http://graphs.mozilla.org/graph.html#tests=[[242,94,12]]&sel=none&displayrange=30&datatype=running 

Looks like ionmonkey landing did make us read more...but the increase caused by webrtc is more terrifying, it added 1mb of IO, in addition to 500K caused by ionmonkey.

I'm still not convinced that this accounts for increased long tail in ff18.

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Taras Glek (:taras) from comment #9)
> Looks like ionmonkey landing did make us read more...but the increase caused
> by webrtc is more terrifying, it added 1mb of IO, in addition to 500K caused
> by ionmonkey.

If IO includes the executable itself - this may not be surprising, as codesize grew significantly.  libxul.so over the weekend grew by 1.8MB (stripped), from 30.x to 32MB, or around 6% (smaller % of the entire load size, but xul is the majority of Firefox).  This was from comparing linux32 nightlies from 
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/10/2012-10-06-13-47-17-mozilla-central/
and
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/10/2012-10-09-03-05-47-mozilla-central/
after stripping.  If those are debug builds, the difference would be smaller.

So, including code loading, 1MB+ increase is to be expected.  I imagine ionmonkey also bumped the code size noticably.

WebRTC should be doing little at startup other than a few static initializers.

> I'm still not convinced that this accounts for increased long tail in ff18.

Comment 11

[brendan c]

Attachment: Analysis of SIMPLE_MEASURES_FIRSTPAINT from Telemetry, FF17 vs FF18 (nightly)

Analysis of SIMPLE_MEASURES_FIRSTPAINT from Telemetry did not show evidence of performance regression in FF18 nightly builds, including on or around 2012.09.11. This does not rule out a regression, but there is not conclusive evidence in this data, and the data examined was not adequate to isolate a regression in any particular build.

Comment 12

[(dormant account)]

My evidence for a regression was slim and could've been caused by a slightly different set of data coming in due to increased level of telemetry reporting(bug 792065). Looks like there is no evidence of a significant regression from telemetry data analyzed by Brendan.