Closed Bug 1215553 Opened 9 years ago Closed 7 years ago

[New-Homescreen] fullyLoaded time regresses significantly vs. verticalhome

Categories

(Firefox OS Graveyard :: Gaia::Homescreen, defect, P1)

Product:
Firefox OS Graveyard
Component:
Gaia::Homescreen
Platform:
ARM
Gonk (Firefox OS)
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: cwiiis, Assigned: cwiiis)

References

Details

(Keywords: perf, regression, Whiteboard: [systemsfe])

Attachments

(1 file)

[gaia] Cwiiis:bug1215553-new-homescreen-startup-fullyloaded > mozilla-b2g:master
46 bytes, text/x-github-pull-request
gmarty
: review+
Details | Review
So we fixed visuallyLoaded, but on restart tests, new homescreen's fullyLoaded time regresses significantly over verticalhome. verticalhome: $ raptor test restart-b2g --runs 30 [Restart B2G] Results from verticalhome.gaiamobile.org | Metric | Mean | Median | Min | Max | StdDev | p95 | | --------------------- | --------- | --------- | ------ | ------ | ------- | ------ | | navigationLoaded | 11833.333 | 11603.500 | 10935 | 13765 | 799.424 | 13622 | | navigationInteractive | 11838.200 | 11607 | 10940 | 13770 | 799.350 | 13625 | | visuallyLoaded | 14432.900 | 14260.500 | 13324 | 16379 | 827.645 | 16173 | | contentInteractive | 14437.933 | 14266.500 | 13329 | 16384 | 827.619 | 16178 | | fullyLoaded | 14869.767 | 14668 | 13543 | 16769 | 903.046 | 16573 | | rss | 45.124 | 44.197 | 43.754 | 52.492 | 1.981 | 50.234 | | uss | 25.908 | 24.968 | 24.520 | 33.281 | 1.984 | 31.023 | | pss | 32.355 | 31.474 | 29.856 | 39.859 | 2.045 | 37.604 | homescreen: $ raptor test restart-b2g --runs 30 --homescreen homescreen [Restart B2G] Results from homescreen.gaiamobile.org | Metric | Mean | Median | Min | Max | StdDev | p95 | | --------------------- | --------- | --------- | ------ | ------ | -------- | ------ | | navigationLoaded | 11848.767 | 11497.500 | 10912 | 13917 | 915.172 | 13714 | | navigationInteractive | 12099.933 | 11723 | 11197 | 14106 | 897.683 | 13925 | | visuallyLoaded | 13246.133 | 12705.500 | 12198 | 15329 | 1030.765 | 15310 | | contentInteractive | 13251.100 | 12710 | 12203 | 15334 | 1030.794 | 15315 | | fullyLoaded | 22093.367 | 21640 | 20502 | 24711 | 1297.404 | 24623 | | rss | 48.064 | 47.520 | 46.746 | 50.336 | 1.139 | 50.246 | | uss | 28.646 | 28.099 | 27.320 | 30.918 | 1.134 | 30.840 | | pss | 33.804 | 33.287 | 32.214 | 36.076 | 1.128 | 35.996 | Let's fix this. I'll allow myself a very slight regression over verticalhome (let's say 2 seconds), but let's see if we can just get it outright better. I expect this will involve some changes in gaia-container as well as homescreen.
I've now optimised gaia-container slightly and optimised loading in homescreen, and gotten these results (taken over 10 runs because 30 takes ages and it's the end of the day): verticalhome: | Metric | Mean | Median | Min | Max | StdDev | p95 | | --------------------- | --------- | --------- | ------ | ------ | ------- | ------ | | navigationLoaded | 12056.700 | 11779 | 11130 | 13931 | 900.793 | 13931 | | navigationInteractive | 12061.800 | 11784 | 11135 | 13936 | 900.801 | 13936 | | visuallyLoaded | 14793.900 | 14514.500 | 13999 | 16473 | 864.862 | 16473 | | contentInteractive | 14799.200 | 14519.500 | 14004 | 16478 | 864.763 | 16478 | | fullyLoaded | 15315.400 | 15141.500 | 14338 | 16975 | 946.556 | 16975 | | pss | 32.106 | 32.409 | 30.258 | 33.506 | 1.271 | 33.506 | | rss | 45.294 | 45.228 | 44.309 | 46.512 | 0.867 | 46.512 | | uss | 25.956 | 25.889 | 24.945 | 27.082 | 0.868 | 27.082 | homescreen with no changes: | Metric | Mean | Median | Min | Max | StdDev | p95 | | --------------------- | --------- | --------- | ------ | ------ | -------- | ------ | | navigationLoaded | 12663.300 | 13298.500 | 10960 | 13693 | 1068.757 | 13693 | | navigationInteractive | 12921.500 | 13547 | 11224 | 13974 | 1084.618 | 13974 | | visuallyLoaded | 13961.700 | 14433.500 | 12007 | 15522 | 1226.801 | 15522 | | contentInteractive | 13966.800 | 14439 | 12012 | 15528 | 1227.027 | 15528 | | fullyLoaded | 22769.300 | 23214.500 | 20398 | 24650 | 1642.905 | 24650 | | uss | 29.791 | 29.102 | 28.406 | 32.355 | 1.333 | 32.355 | | rss | 49.259 | 48.566 | 47.875 | 51.812 | 1.333 | 51.812 | | pss | 34.957 | 34.164 | 33.406 | 37.138 | 1.476 | 37.138 | homescreen with optimisations: | Metric | Mean | Median | Min | Max | StdDev | p95 | | --------------------- | --------- | --------- | ------ | ------ | -------- | ------ | | navigationLoaded | 11901.500 | 11519.500 | 10898 | 13970 | 985.680 | 13970 | | navigationInteractive | 12169.700 | 11724.500 | 11089 | 14248 | 985.704 | 14248 | | visuallyLoaded | 13076.200 | 12541.500 | 11885 | 15026 | 1032.980 | 15026 | | contentInteractive | 13081.100 | 12546.500 | 11890 | 15031 | 1032.637 | 15031 | | fullyLoaded | 18947.800 | 18311 | 17714 | 21431 | 1257.444 | 21431 | | uss | 31.907 | 31.848 | 29.848 | 34.422 | 1.402 | 34.422 | | rss | 51.346 | 51.276 | 49.293 | 53.855 | 1.400 | 53.855 | | pss | 37.234 | 37.543 | 34.616 | 39.702 | 1.480 | 39.702 | That's a pretty nice saving of around half a second on visuallyLoaded and 3 seconds on fullyLoaded (looking at p95). Still behind verticalhome, but beating visuallyLoaded handily and not a million miles away from fullyLoaded. With the current loading mechanism, if we load all apps at the same time (instead of progressively, which allows the user to interact while we're still loading apps), I think we could expect to cut off another second from fullyLoaded, but we'd then of course sacrifice the visuallyLoaded time. Let's get these changes in, I'm going to have to have a think about where to go from here... Possibly web workers, but that would be a considerable refactor :/
hmm, probably shouldn't be looking at p95 actually, that's not what I thought it meant... Anyway, we're about 4 seconds away from verticalhome, which is still 2 seconds slower than I'd like.
Chris, you may want to update @mozilla/raptor to 3.1.0 - we updated the p95 to "upper bound" which is more reliable. Also, unfortunately with stdev that high I doubt you can draw any reasonable conclusion out of 10 runs. In order to check that: 1) remove metrics.ldjson from the directory you're in 2) run raptor on homescreen master 3) run raptor on homescreen with the patch 4) run raptor-compare ./metrics.ldjson The results marked with an asterisk are statistically significant. [0] https://www.npmjs.com/package/raptor-compare
Comment on attachment 8676222 [details] [review] [gaia] Cwiiis:bug1215553-new-homescreen-startup-fullyloaded > mozilla-b2g:master As well as taking advantage of the recent gaia-container updates, this also batches updates in pages after visual load, and freezes the container while adding them to cut down on the amount of relayouting. I think this is a nice trade-off of maintaining user interaction and improving performance. I'm still sure we can do better, though.
Attachment #8676222 - Flags: review?(gmarty)
raptor-compare just quits on start with '[TypeError: undefined is not a function]' for me, unfortunately... Unpatched, 30 tries: | Metric | Mean | Median | Min | Max | StdDev | p95 | | --------------------- | --------- | --------- | ------ | ------ | -------- | ------ | | navigationLoaded | 12562.933 | 13060 | 10784 | 14113 | 1022.639 | 14043 | | navigationInteractive | 12819.267 | 13309.500 | 11039 | 14292 | 1002.964 | 14239 | | visuallyLoaded | 13804.300 | 14164 | 12021 | 15408 | 979.727 | 15193 | | contentInteractive | 13809.233 | 14168.500 | 12025 | 15412 | 979.740 | 15198 | | fullyLoaded | 22513.800 | 22592.500 | 20178 | 24769 | 1291.617 | 24363 | | uss | 29.364 | 28.582 | 28.035 | 32.262 | 1.283 | 31.996 | | pss | 34.520 | 34.106 | 32.824 | 37.979 | 1.388 | 37.049 | | rss | 48.830 | 48.073 | 47.500 | 51.754 | 1.289 | 51.465 | Patched, 30 tries: | Metric | Mean | Median | Min | Max | StdDev | p95 | | --------------------- | --------- | --------- | ------ | ------ | -------- | ------ | | navigationLoaded | 12452.533 | 13140 | 10035 | 13761 | 1097.483 | 13677 | | navigationInteractive | 12708.633 | 13334.500 | 10359 | 13942 | 1070.533 | 13933 | | visuallyLoaded | 13679.567 | 14111 | 11900 | 14941 | 954.659 | 14815 | | contentInteractive | 13684.533 | 14116 | 11901 | 14946 | 955.009 | 14821 | | fullyLoaded | 19750.833 | 19971.500 | 17894 | 21977 | 1143.989 | 21519 | | uss | 31.314 | 31.518 | 27.836 | 37.320 | 2.481 | 35.180 | | pss | 36.922 | 37.182 | 33.636 | 43.024 | 2.444 | 40.176 | | rss | 50.765 | 50.967 | 47.246 | 56.777 | 2.491 | 54.645 | I'm uncertain of what we can statistically conclude from this, however, it seems like a pretty clear general improvement and for what it's worth, it feels faster.
Comment on attachment 8676222 [details] [review] [gaia] Cwiiis:bug1215553-new-homescreen-startup-fullyloaded > mozilla-b2g:master r+'ed, but I left a couple of comments on Github and as before, I'd really prefer that the visual loading event were triggered when a full page of icons + 1 row from the next page are visible. It's not fully loaded if there are elements not visible. But it's just a minor detail.
Attachment #8676222 - Flags: review?(gmarty) → review+
(In reply to Guillaume Marty [:gmarty] from comment #7) > Comment on attachment 8676222 [details] [review] > [gaia] Cwiiis:bug1215553-new-homescreen-startup-fullyloaded > > mozilla-b2g:master > > r+'ed, but I left a couple of comments on Github and as before, I'd really > prefer that the visual loading event were triggered when a full page of > icons + 1 row from the next page are visible. It's not fully loaded if there > are elements not visible. But it's just a minor detail. We do actually do this, but platform means that the icons don't load in synchronously - if you scroll down while it looks like the last row is empty, you'll see the icon subtitles are visible. I don't know what we can do to fix this (tbh, I don't even know why the icons above that row all appear to load synchronously...)
Merged: https://github.com/mozilla-b2g/gaia/commit/32d827a70af90a05918f234e5b16b35d5d2a07e8 Not marking this as fixed yet, but with this optimisation, this is slightly lower priority now. There's still a second or two that I think could be gained by being more careful with DOM operations, but it might be hard to eke out those gains without sacrificing abstraction.
> raptor-compare just quits on start with '[TypeError: undefined is not a function]' for me, unfortunately... raptor-compare was not compatible with node 0.12.7 - :stas pushed an updated to npm which makes it compatible.
I think the next big startup time win will come from bug 1220186 - I think they'll be a second or two there, which would put us in a reasonable place.
Depends on: 1220186
Bug 1224136 saves ~2 seconds on fullyLoaded time, which significantly closes the gap between homescreen and verticalhome. verticalhome still has an ~1 second advantage on fullyLoaded, but a ~2 second deficit on visuallyLoaded.
Depends on: 1224136
Hi Eli, if you have the time, do you think you could run some homescreen/verticalhome tests on master to see where we are now with this? Obviously I have my own results on the Z3C, but I don't trust my bias or ability :) It would get good to get Flame results too. Not at all urgent if you're busy.
Flags: needinfo?(eperelman)
According to Raptor for Homescreen [1], fullyLoaded sits at around 13-14 seconds higher than visuallyLoaded. [1] http://mzl.la/1kCoa2s
Flags: needinfo?(eperelman)
Looking at recent results, we're still behind verticalhome, though I don't think it's too significant at this point. Leaving this open to track further optimisation. Things that strike me: - We don't enforce loading order strongly enough, so some things can load out-of-order and cause animations and things. - There are still more reflows than are necessary during startup (bug 1220186) - We should possibly consider making cached icon-image loading asynchronous to icon-order loading - We should possibly consider using workers to make icon metadata loading asynchronous to page layout/rendering
Keywords: regression
Priority: -- → P1
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: