807906 - Incrementally load views in calendar to reduce load times.

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Calendar, defect, P3)

Description

[James Lal [:lightsofapollo]]

There is is a core of files which must be loaded fairly early (store/*, calendar.js, some of the utility files) but nearly all the views can be lazily loaded and some of the utilities and models.

-- 

Blocking rational:

Loading calendar takes around 4s (4665ms on my unagi) there are a few obvious steps to improve this and this is one the most important ones.

Comment 1

[Josh Carpenter [:jcarpenter]]

cc'ing Peter for awareness of this as a performance issue.

Comment 2

[Marijn Haverbeke]

I'll take this. (If someone with privileges comes by, please assign it to me.)

Comment 3

[Jason Smith [:jsmith]]

Btw Marijn - I'm working on getting you edit bug access. You should see your account change soon to have higher privileges.

Comment 4

[James Lal [:lightsofapollo]]

I want to ensure we are very data-driven here I know this will improve load-time performance but I don't know by how much. Lets get some gecko-profiles here first.

My recommendation would be to attempt to reduce the amount of files loaded in views/*.js first and see what impact that has, those are the easiest to lazy load. (settings, advanced settings, day view, week view, modify event, modify account)

My ideal situation is to have a js object that has the list of js & css dependencies of each "package"/"state" and then have a simple lightweight api to load those dependencies (and skip loading those that have already been done).

There are also HTML parse time reduction steps we can take in combination with the lazy loading of views but we should only pursue that after we have some better measurements of what benefits we can expect.

The "app" (app.js) already has logic for lazy initializing the views so it should be very easy to enhance that to also load the associated files.

Comment 5

[Marijn Haverbeke]

I think this is looking pretty bleak. The amount of code for the views is small -- around 3500 lines -- and a large chunk of that is needed to display the starting screen anyway. The amount of css and images that can be delayed is also quite small.

Someone who's deeply familiar with the code might be able to come up with some tricks to speed up loading at least a quarter, but I'm finding the large amount indirection and entanglement hampering my attempts at understanding what going on.

I'm going to back off from this bug for now, and see if I can make more headway somewhere else.

Comment 6

[Kevin Grandon :kgrandon]

I'm new to this, but this seems like an interesting one. If no one objects, I'll gladly take a stab at it :)

Comment 7

[Kevin Grandon :kgrandon]

At first glance it appears that a simple asynchronous dependency loader might help us.

I'm noticing the following on first load: 111 requests @ 469.8kb.

I would imagine that we could cut down on about half of all views, models, stores, and templates on load. This would probably be around the ballpark of ~20 fewer http requests and 100kb off the intial load.

Comment 8

[Kevin Grandon :kgrandon]

First pull request: https://github.com/mozilla-b2g/gaia/pull/6570

There's a lot more we can do if this is an acceptable approach.

Comment 9

[Dylan Oliver [:doliver]]

Hi Kevin,

Can you create an attachment with a pointer to the PR and request a review from James Lal? He'll be back from vacation tomorrow and can take a look at it.

Comment 10

[Kevin Grandon :kgrandon]

Attachment: Pre-dependency profile

Comment 11

[Kevin Grandon :kgrandon]

Attachment: Post-dependency profile

Comment 12

[James Lal [:lightsofapollo]]

In general we are looking for overall start-up times to go down you can also turn on a pref in developer settings (I can't remember what it is called exactly) that will add a green colored timer that displays start up times.

Comment 13

[James Lal [:lightsofapollo]]

OK, I took a quick look and it looks like JS parse time went down (65 old vs 48 new) it is a fairly small difference but it helps. The best test here is probably just raw startup time with the green timer (which is less confusing then the profiler).

It may not have been obvious but to read these files you need the cleopatra UI (https://github.com/bgirard/cleopatra) you can then narrow the range in the timebar (click and drag) to see a specific time range. I will do some more manual tests and report back here.

Also- Looking at this closely it seems like the next major target would be to unblock the UI from waiting on the db being ready (accounts, calendars, settings). It could be a bit tricky but this fits in closely with your DI changes because we only need those things in some views and prior to sync.

Comment 14

[Kevin Grandon :kgrandon]

B2G desktop startup numbers were a wash. Here's a few consecutive runs on my unagi:

Phone before DI: 2681, 2622, 2875, 2625, 2611, 2663, 2615, 2727, 2740, 2779
Phone after DI: 2354, 2474, 2609, 2223, 2213, 3133, 2473, 2202, 2532, 2704

I would get the occasional slow load with either solution (3500 ms+). This is about a 100-200ms improvement with the current solution, and I think it is a good road to go down. 

Once we get the pull request polished, I'd like to merge it in and continue to make performance improvements with the model layer.

Comment 15

[James Lal [:lightsofapollo]]

Sounds good. I think the spike is due to the initial database "load" which is an important area to optimize. I think that will be easier to lazy load once we complete views and models. From there we can begin to lazy load the stores.

Shorter term, I think it would be very easy to lazy load the worker/* and provider/caldav.js (caldav can depend on advanced settings?).

Keep in mind that as we saw in settings, html has an impact on load times as well. Now that we lazy load the views we could easily use the same technique (commenting out the contents of the view) in combination with your DI patch. That may yield equal or greater benefits to lazy loading a few more small files.

Comment 16

[Kevin Grandon :kgrandon]

Ten runs after templating optimiziation on my unagi: 2335, 2257, 2502, 2200, 2381, 2463, 2258, 2280, 2234, 2315

Before average: 2,694
Avg now: 2,323

Slight improvement. More to go.

Comment 17

[James Lal [:lightsofapollo]]

Not bad, if we can fully async calendar/account loading we could easily win another 100-200ms but I think that should be another bug, I will open it now...

Comment 18

[Kevin Grandon :kgrandon]

Pull request: https://github.com/mozilla-b2g/gaia/pull/6570

Comment 19

[James Lal [:lightsofapollo]]

r+ on GH nice work Kevin, landing.

Comment 20

[James Lal [:lightsofapollo]]

A little background we have not lazy loaded everything yet but we do lazy load all the views not required for the initial screen and we have a new system to lazy load everything else as possible. I am going to close this out and we can continue work in other bugs for the other pieces.

Comment 21

[Kevin Grandon :kgrandon]

Another pull request to move to string concatenation: https://github.com/mozilla-b2g/gaia/pull/6758