Closed Bug 814981 Opened 11 years ago Closed 10 years ago
Initial app launch times on core apps is slow
843.25 KB, video/mp4
1.44 MB, video/mp4
464.45 KB, video/mp4
591.38 KB, video/mp4
591.70 KB, video/mp4
755.10 KB, video/mp4
899.66 KB, video/mp4
What makes it feel slow/broken? Almost every app feels slow to launch on the initial launch (initial launch is defined as first time launching after a device reset). Many apps take 5 seconds to launch, and some take much longer (E-Mail, Cost Control, Calendar). Did it prevent you from doing what you wanted? Why? It slows me down and gives me a bad first impression as a user. It sets my expectations that the OS is going to be slow. How does this make you feel? [ ] :) I feel happy about it [ ] :| Meh [X] :( I'm upset [ ] >:O I'm angry Device: Unagi, Nov. 22 Nightly. Details: Comparing with our reference competitor platform (Android ICS on Otoro), the typical launch times are as follows. The first number = initial launch time, the second number = subsequent launch time. Our goal should be to do as well as ICS for initial launch times. Benchmarks B2G on Unagi Settings - 4.1 / .7s Browser - 2.5s / .7s Camera - 4.7s / 1.7s Dialer - 5s / .7s Contacts - 5.7 / .7s E-Mail - 9-11.5 / .7s (GMail/8 Messages in inbox/Wi-Fi) Cost Control - 12.7s / .7s Calendar - 11.3s / 1s Clock - 4.8 / 1.5s SMS - 3s / 1s Gallery - 4s / 1.5s Music - 5s / 1s Video - 5s / 1s Calculator - 3.4s / 1s FM Radio - 3.4s / 1s Marketplace - 4.5-6s / 1.4s Android ICS on Otoro Settings - .5s / .5s Browser - 3s / .7s Camera - 3.5s / 2.6s Dialer - 2s / .7s Contacts - 1.3 / .7s E-Mail - 3 / .7s Cost Control - N/A Calendar - 1.9s / .7s Clock - 2.5 / 1.1s SMS - ?? / .7s Gallery - 1.7s / .7s Music - 1.5s / .7s Video - 1s / .7s Calculator - 1.5s / .7s FM Radio - 1.6s / 1s Marketplace - N/A Bonus: can you attach a video of the problem? Yes
We have to be very careful here to compare cold-launch times to cold-launch times. ICS, and soon b2g, will launch some things in the background to improve startup time. How are you measuring launch time? That is, what series of steps are you following to measure, how do you define "launched", and how are you measuring times.
Summary: [Gaia][perf][uxtrust] Initial app launch times on core apps is slow → [Gaia] Initial app launch times on core apps is slow
I am comparing cold launch with cold launch. The first column of numbers is cold launch time, the second column is subsequent launch. "cold launch" is after you reset the device, unlock the device, and launch the app. "launched" is defined as from the moment the icon is tapped to the moment the app is interactible. I am timing with a stopwatch, so the times will have an error of ~200ms or less. Question: should a bug per app be created for launch times or is one bug sufficient?
Nominating for Basecamp Blocking+. Cold app launch times are currently much longer than Android on Otoro, with the exception of the browser, and is contributing to a bad user experience. However, we need a metric for determining when this bug is considered fixed. Please advise...
blocking-basecamp: --- → ?
(In reply to Peter La from comment #2) > I am comparing cold launch with cold launch. The first column of numbers is > cold launch time, the second column is subsequent launch. > > "cold launch" is after you reset the device, unlock the device, and launch > the app. That's not going to result in an apples to apples comparison in general, because apps can be prelaunched. (We'll be investigating this for b2g in bug 815560.) This also doesn't account for app assets being in disk cache. Better is to kill any processes related to the app being tested in between runs, and do multiple trials and throw out outlier numbers. > "launched" is defined as from the moment the icon is tapped to the moment > the app is interactible. What does "interactible" mean? > Question: should a bug per app be created for launch times or is one bug > sufficient? One per app is better for tracking purposes.
(In reply to Chris Jones [:cjones] [:warhammer] from comment #11) > (In reply to Peter La from comment #2) > > I am comparing cold launch with cold launch. The first column of numbers is > > cold launch time, the second column is subsequent launch. > > > > "cold launch" is after you reset the device, unlock the device, and launch > > the app. > > That's not going to result in an apples to apples comparison in general, > because apps can be prelaunched. (We'll be investigating this for b2g in > bug 815560.) Although I should add that an apples to oranges comparison is also useful, since that's competitive parity.
Hi Chris, Thanks for the quick responses. Interactible means you can start using the app - it is ready to respond to user events. Often an app will launch and render, but does not respond to user events right away. As for doing proper cold launches, I don't currently know how to kill process related to each app, or what those processes are. I would need help with this, or have the individual bug fixers do the more rigorous testing per app to get definitive numbers. I can file a separate bug for each app.
That definition sounds pretty hard to apply consistently to me. I'd recommend we look at three things - when the UI first appears - when the UI first "looks like" it can be interacted with, for example when the first buttons/links appear - if you can come up with a consistently-applicable definition, when those buttons can actually be interacted with As for "properly" measuring cold-start, I changed my mind :). Let's go with apples-to-oranges. That's what users will see. Let's make sure to reboot both OSes in between each series of measurements though, and wait a minute or so before starting to measure. Otherwise your data will be hard to reproduce.
I considerate this as a Meta Bug, but the solution will be different from an app to another and the measurement must follow Chris suggestion. Please add dependents bug to this.
blocking-basecamp: ? → ---
I'm creating separate bugs right now. Decided not to wait until David has his new test numbers in. David, can you add your numbers to all the separate bugs once you've got them? (see all blocking bugs) Thanks.
Note on method used for timing: 1) Restart the device 2) Unlock the device 3) Wait 1 minute for the device to settle 4) Using a stopwatch program (I used iPhone), start the timer exactly when you tap on the application's icon 5) Pick an interactible element on the first screen that is shown at app launch (best if it's a button), and when you see this element, start tapping on it repeatedly. 6) As soon as the button responds, you can stop the timer. It may take several tries to get it right. You can practice without doing steps 1-3 first, then do the real timing with a device restart. Furthermore, you may want to record 3 different times based on what Chris Jones outlined above: 1) when you first see the UI, 2) when the UI first looks interactible, 3) when the UI actually becomes interactible.
Priority: P1 → --
Summary: [Gaia] Initial app launch times on core apps is slow → Initial app launch times on core apps is slow
Whiteboard: perf, ux-trust → UX-P1
OS: Mac OS X → Gonk (Firefox OS)
Hardware: x86 → ARM
These are some informal updated startup times I took last week on Jan. 14th build for Otoro. Unfortunately I don't have numbers for Otoro from October, so I am listing them next to the Unagi numbers from October. Timing method used was the same for both: 1 Restart device 2 Unlock device 3 Wait 30 seconds 4 Tap icon and start stopwach 5 Wait until App can be interacted with and stop stopwatch (this means the UI has fully rendered and no visible loading is taking place). The process was done 2 times for each app. -- Oct. Unagi vs Jan. 14 Otoro Numbers -- App Startup Times App Unagi/Oct Otoro/Jan Settings - 4.1 / 3.0 -> 27% Browser - 2.5s / 1.5 -> 40% Camera - 4.7s / 5 Dialer - 5s / 2.3 -> 54% Contacts - 5.7 / 3.6 -> 36% E-Mail - 9-11.5 / 5.2 (no accounts setup) Cost Control - 12.7s / 1.8 -> 78% Calendar - 11.3s / 3.3 -> 70% Clock - 4.8 / 2.7 -> 43% SMS - 3s / 2.2 -> 27% Gallery - 4s / 2.7 -> 32% Music - 5s / 2.8 -> 44% Video - 5s / 5 FM Radio - 3.4s / 2 -> 41% Marketplace - 4.5-6s / 1.9 (no connectivity) Return to Homescreen - 1-3 sec typical, sometimes 3+ / Currently: ~.5s or less --
Whiteboard: UX-P1, [perf_tsk] → UX-P1, [FFOS_perf]
This is a meta bug and we only block on bugs that are actionable.
blocking-b2g: tef? → ---
Please read comment 19. Don't renom this one. Thanks.
blocking-b2g: tef? → ---
Whiteboard: UX-P1, [FFOS_perf] → UX-P1, [FFOS_perf], PRODUCT-PERF
cc Arun for additional metrics.
Per comment 19 -- if this bug is not actionable, is there a reason this bug is still open with a NEW status? I think something like this should be marked RESOLVED WONTFIX or INVALID or similar so the bug DB can stay relevant. Or is adding the keyword "meta" enough and people should just know to filter those bugs out if they're looking to contribute?
Hi Darius. This is very common in Mozilla development to have a meta bug like this which identifies a set of individually actionable items of work, and can itself be closed once all of those items are closed. If you see a meta-bug like this, check out each of the dependent bugs to see which are unowned. Feel free to email me directly if you want more info.
Whiteboard: UX-P1, [FFOS_perf], PRODUCT-PERF → UX-P1, [FFOS_perf], PRODUCT-PERF c=
Whiteboard: UX-P1, [FFOS_perf], PRODUCT-PERF c= → UX-P1, PRODUCT-PERF [c= ]
FIXED with all the performance work done in the last month.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Whiteboard: UX-P1, PRODUCT-PERF [c= ] → UX-P1, PRODUCT-PERF [c= p= s=2013.10.25 u=]
You need to log in before you can comment on or make changes to this bug.