Last Comment Bug 798130 - Regression in startup in 18.
: Regression in startup in 18.
Status: RESOLVED INVALID
[Snappy]
:
Product: Firefox
Classification: Client Software
Component: General (show other bugs)
: unspecified
: x86_64 Windows 7
: -- normal (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on: 799227
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-04 16:02 PDT by (dormant account)
Modified: 2012-10-23 16:05 PDT (History)
24 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
screen cap of SIMPLE_MEASURES_FIRSTPAINT histogram (126.19 KB, image/png)
2012-10-05 14:30 PDT, brendan c
no flags Details
screen cap of SIMPLE_MEASURES_FIRSTPAINT evolution (136.56 KB, image/png)
2012-10-05 14:36 PDT, brendan c
no flags Details
Analysis of SIMPLE_MEASURES_FIRSTPAINT from Telemetry, FF17 vs FF18 (nightly) (71.46 KB, application/pdf)
2012-10-23 16:00 PDT, brendan c
no flags Details

Description (dormant account) 2012-10-04 16:02:22 PDT
The long tail is pretty bad in ff18 compared to previous release http://is.gd/S7x4yv
Comment 1 (dormant account) 2012-10-04 16:05:24 PDT
According to evolution looks like things started trending up on sep11th. 
http://is.gd/1t5pIv
Comment 2 Nathan Froyd [:froydnj] 2012-10-04 19:12:37 PDT
That would probably be the IonMonkey merge, though dev.tree-management doesn't appear to have any Ts regressions.  There are some ~5% Tp5 regressions around that time or shortly thereafter, though.
Comment 3 (dormant account) 2012-10-04 19:23:50 PDT
(In reply to Nathan Froyd (:froydnj) from comment #2)
> That would probably be the IonMonkey merge, though dev.tree-management
> doesn't appear to have any Ts regressions.  There are some ~5% Tp5
> regressions around that time or shortly thereafter, though.

It lines up with ionmonkey, but this regression is especially obvious on the tail and I can't imagine something cpu-bound could affecting startups >30s.
Comment 4 David Anderson [:dvander] 2012-10-04 22:42:32 PDT
I don't appear to have access to that site so I can't see the data. It would indeed be strange if it was IonMonkey - we have limits in place to curb the compilation process from exploding and haven't seen evidence of that in the wild yet. I guess it increased the binary size by some non-trivial amount, I think around 1MB?
Comment 5 Matthew N. [:MattN] (behind on reviews) 2012-10-04 22:59:02 PDT
(In reply to David Anderson [:dvander] from comment #4)
> I don't appear to have access to that site so I can't see the data.

Login with Persona using any email address.
Comment 6 brendan c 2012-10-05 14:30:43 PDT
Created attachment 668621 [details]
screen cap of SIMPLE_MEASURES_FIRSTPAINT histogram
Comment 7 brendan c 2012-10-05 14:36:11 PDT
Created attachment 668625 [details]
screen cap of SIMPLE_MEASURES_FIRSTPAINT evolution

from http://is.gd/1t5pIv
Comment 8 brendan c 2012-10-05 14:40:50 PDT
Taras, I want to make sure we are on the same page (literally looking at the same telemetry page)--

When I follow your first link ( http://is.gd/S7x4yv ) the highest startup times (>30s) are better in 18 than 17. (see screen cap https://bugzilla.mozilla.org/attachment.cgi?id=668621 )

And looking at http://is.gd/1t5pIv (see https://bugzilla.mozilla.org/attachment.cgi?id=668625 ) Sept. 11 doesn't look especially interesting (could just be noise); in fact, the series seems to dip locally after that day...

Please let me know if ended up looking at the wrong graphs, or if I'm missing the feature you are concerned about, interpreting something differently, etc..
Comment 9 (dormant account) 2012-10-08 12:04:22 PDT
(In reply to Taras Glek (:taras) from comment #1)
> According to evolution looks like things started trending up on sep11th. 
> http://is.gd/1t5pIv

Sep 11 regression is also confirmed by our io tracker http://graphs.mozilla.org/graph.html#tests=[[242,94,12]]&sel=none&displayrange=30&datatype=running 

Looks like ionmonkey landing did make us read more...but the increase caused by webrtc is more terrifying, it added 1mb of IO, in addition to 500K caused by ionmonkey.

I'm still not convinced that this accounts for increased long tail in ff18.
Comment 10 Randell Jesup [:jesup] 2012-10-09 12:11:33 PDT
(In reply to Taras Glek (:taras) from comment #9)
> Looks like ionmonkey landing did make us read more...but the increase caused
> by webrtc is more terrifying, it added 1mb of IO, in addition to 500K caused
> by ionmonkey.

If IO includes the executable itself - this may not be surprising, as codesize grew significantly.  libxul.so over the weekend grew by 1.8MB (stripped), from 30.x to 32MB, or around 6% (smaller % of the entire load size, but xul is the majority of Firefox).  This was from comparing linux32 nightlies from 
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/10/2012-10-06-13-47-17-mozilla-central/
and
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/10/2012-10-09-03-05-47-mozilla-central/
after stripping.  If those are debug builds, the difference would be smaller.

So, including code loading, 1MB+ increase is to be expected.  I imagine ionmonkey also bumped the code size noticably.

WebRTC should be doing little at startup other than a few static initializers.

> I'm still not convinced that this accounts for increased long tail in ff18.
Comment 11 brendan c 2012-10-23 16:00:44 PDT
Created attachment 674422 [details]
Analysis of SIMPLE_MEASURES_FIRSTPAINT from Telemetry, FF17 vs FF18 (nightly)

Analysis of SIMPLE_MEASURES_FIRSTPAINT from Telemetry did not show evidence of performance regression in FF18 nightly builds, including on or around 2012.09.11. This does not rule out a regression, but there is not conclusive evidence in this data, and the data examined was not adequate to isolate a regression in any particular build.
Comment 12 (dormant account) 2012-10-23 16:05:18 PDT
My evidence for a regression was slim and could've been caused by a slightly different set of data coming in due to increased level of telemetry reporting(bug 792065). Looks like there is no evidence of a significant regression from telemetry data analyzed by Brendan.

Note You need to log in before you can comment on or make changes to this bug.