All users were logged out of Bugzilla on October 13th, 2018

Initial app launch times on core apps is slow

RESOLVED FIXED

Status

RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: pla, Unassigned)

Tracking

({meta, perf})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: UX-P1, PRODUCT-PERF [c= p= s=2013.10.25 u=])

Attachments

(7 attachments)

(Reporter)

Description

6 years ago
What makes it feel slow/broken?

Almost every app feels slow to launch on the initial launch (initial launch is defined as first time launching after a device reset).  Many apps take 5 seconds to launch, and some take much longer (E-Mail, Cost Control, Calendar).

Did it prevent you from doing what you wanted? Why?

It slows me down and gives me a bad first impression as a user.  It sets my expectations that the OS is going to be slow.

How does this make you feel?

[ ]  :)  I feel happy about it
[ ]  :|  Meh
[X]  :(  I'm upset
[ ] >:O  I'm angry

Device: Unagi, Nov. 22 Nightly.

Details:

Comparing with our reference competitor platform (Android ICS on Otoro), the typical launch times are as follows.  The first number = initial launch time, the second number = subsequent launch time.

Our goal should be to do as well as ICS for initial launch times.

  Benchmarks

  B2G on Unagi

  Settings     - 4.1    / .7s
  Browser      - 2.5s   / .7s
  Camera       - 4.7s   / 1.7s
  Dialer       - 5s     / .7s
  Contacts     - 5.7    / .7s
  E-Mail       - 9-11.5 / .7s (GMail/8 Messages in inbox/Wi-Fi)
  Cost Control - 12.7s  / .7s
  Calendar     - 11.3s  / 1s
  Clock        - 4.8    / 1.5s
  SMS          - 3s     / 1s
  Gallery      - 4s     / 1.5s
  Music        - 5s     / 1s
  Video        - 5s     / 1s
  Calculator   - 3.4s   / 1s
  FM Radio     - 3.4s   / 1s
  Marketplace  - 4.5-6s / 1.4s

  Android ICS on Otoro

  Settings     - .5s    / .5s
  Browser      - 3s     / .7s
  Camera       - 3.5s   / 2.6s
  Dialer       - 2s     / .7s
  Contacts     - 1.3    / .7s
  E-Mail       - 3      / .7s
  Cost Control - N/A
  Calendar     - 1.9s   / .7s
  Clock        - 2.5    / 1.1s
  SMS          - ??     / .7s
  Gallery      - 1.7s   / .7s
  Music        - 1.5s   / .7s
  Video        - 1s     / .7s
  Calculator   - 1.5s   / .7s
  FM Radio     - 1.6s   / 1s
  Marketplace  - N/A

Bonus: can you attach a video of the problem?
Yes
(Reporter)

Updated

6 years ago
Blocks: 815163
We have to be very careful here to compare cold-launch times to cold-launch times.  ICS, and soon b2g, will launch some things in the background to improve startup time.

How are you measuring launch time?  That is, what series of steps are you following to measure, how do you define "launched", and how are you measuring times.
Blocks: 797189
Priority: -- → P1
(Reporter)

Updated

6 years ago
Blocks: 814988
Whiteboard: perf, uxtrust → perf, ux-trust
(Reporter)

Updated

6 years ago
Keywords: ux-trust
Summary: [Gaia][perf][uxtrust] Initial app launch times on core apps is slow → [Gaia] Initial app launch times on core apps is slow
(Reporter)

Comment 2

6 years ago
I am comparing cold launch with cold launch.  The first column of numbers is cold launch time, the second column is subsequent launch.

"cold launch" is after you reset the device, unlock the device, and launch the app.

"launched" is defined as from the moment the icon is tapped to the moment the app is interactible.

I am timing with a stopwatch, so the times will have an error of ~200ms or less.

Question: should a bug per app be created for launch times or is one bug sufficient?
(Reporter)

Comment 3

6 years ago
Created attachment 685740 [details]
App Load Time - Calendar
(Reporter)

Comment 4

6 years ago
Created attachment 685742 [details]
App Load Time - Camera
(Reporter)

Comment 5

6 years ago
Created attachment 685743 [details]
App Load Time - Contacts
(Reporter)

Comment 6

6 years ago
Created attachment 685745 [details]
App Load Time - Cost Control
(Reporter)

Comment 7

6 years ago
Created attachment 685746 [details]
App Load Time - Dialer
(Reporter)

Comment 8

6 years ago
Created attachment 685747 [details]
App Load Time - Email
(Reporter)

Comment 9

6 years ago
Created attachment 685749 [details]
App Load Time - Settings
(Reporter)

Comment 10

6 years ago
Nominating for Basecamp Blocking+.

Cold app launch times are currently much longer than Android on Otoro, with the exception of the browser, and is contributing to a bad user experience.

However, we need a metric for determining when this bug is considered fixed.  Please advise...
blocking-basecamp: --- → ?
(In reply to Peter La from comment #2)
> I am comparing cold launch with cold launch.  The first column of numbers is
> cold launch time, the second column is subsequent launch.
> 
> "cold launch" is after you reset the device, unlock the device, and launch
> the app.

That's not going to result in an apples to apples comparison in general, because apps can be prelaunched.  (We'll be investigating this for b2g in bug 815560.)  This also doesn't account for app assets being in disk cache.

Better is to kill any processes related to the app being tested in between runs, and do multiple trials and throw out outlier numbers.

> "launched" is defined as from the moment the icon is tapped to the moment
> the app is interactible.

What does "interactible" mean?

> Question: should a bug per app be created for launch times or is one bug
> sufficient?

One per app is better for tracking purposes.
(In reply to Chris Jones [:cjones] [:warhammer] from comment #11)
> (In reply to Peter La from comment #2)
> > I am comparing cold launch with cold launch.  The first column of numbers is
> > cold launch time, the second column is subsequent launch.
> > 
> > "cold launch" is after you reset the device, unlock the device, and launch
> > the app.
> 
> That's not going to result in an apples to apples comparison in general,
> because apps can be prelaunched.  (We'll be investigating this for b2g in
> bug 815560.)

Although I should add that an apples to oranges comparison is also useful, since that's competitive parity.
(Reporter)

Comment 13

6 years ago
Hi Chris,

Thanks for the quick responses.

Interactible means you can start using the app - it is ready to respond to user events.  Often an app will launch and render, but does not respond to user events right away.

As for doing proper cold launches, I don't currently know how to kill process related to each app, or what those processes are.  I would need help with this, or have the individual bug fixers do the more rigorous testing per app to get definitive numbers.

I can file a separate bug for each app.
That definition sounds pretty hard to apply consistently to me.  I'd recommend we look at three things
 - when the UI first appears
 - when the UI first "looks like" it can be interacted with, for example when the first buttons/links appear
 - if you can come up with a consistently-applicable definition, when those buttons can actually be interacted with

As for "properly" measuring cold-start, I changed my mind :).  Let's go with apples-to-oranges.  That's what users will see.

Let's make sure to reboot both OSes in between each series of measurements though, and wait a minute or so before starting to measure.  Otherwise your data will be hard to reproduce.
I considerate this as a Meta Bug, but the solution will be different from an app to another and the measurement must follow Chris suggestion.
Please add dependents bug to this.
blocking-basecamp: ? → ---
Keywords: meta
(Reporter)

Updated

6 years ago
Depends on: 817048
(Reporter)

Updated

6 years ago
Depends on: 817051
(Reporter)

Updated

6 years ago
Depends on: 817052
(Reporter)

Updated

6 years ago
Depends on: 817054
(Reporter)

Updated

6 years ago
Depends on: 817095
(Reporter)

Updated

6 years ago
Depends on: 817099
(Reporter)

Updated

6 years ago
Depends on: 817109
(Reporter)

Comment 16

6 years ago
I'm creating separate bugs right now.  Decided not to wait until David has his new test numbers in.  David, can you add your numbers to all the separate bugs once you've got them?  (see all blocking bugs)

Thanks.
(Reporter)

Updated

6 years ago
Depends on: 817113
(Reporter)

Updated

6 years ago
Depends on: 817115
(Reporter)

Updated

6 years ago
Depends on: 817116
(Reporter)

Comment 17

6 years ago
Note on method used for timing:

1) Restart the device
2) Unlock the device
3) Wait 1 minute for the device to settle
4) Using a stopwatch program (I used iPhone), start the timer exactly when you tap on the application's icon
5) Pick an interactible element on the first screen that is shown at app launch (best if it's a button), and when you see this element, start tapping on it repeatedly.
6) As soon as the button responds, you can stop the timer.

It may take several tries to get it right.  You can practice without doing steps 1-3 first, then do the real timing with a device restart.

Furthermore, you may want to record 3 different times based on what Chris Jones outlined above: 1) when you first see the UI, 2) when the UI first looks interactible, 3) when the UI actually becomes interactible.
Keywords: ux-trust
Priority: P1 → --
Summary: [Gaia] Initial app launch times on core apps is slow → Initial app launch times on core apps is slow
Whiteboard: perf, ux-trust → UX-P1
(Reporter)

Updated

6 years ago
Depends on: 819065
(Reporter)

Updated

6 years ago
Depends on: 819076
(Reporter)

Updated

6 years ago
Depends on: 819080
(Reporter)

Updated

6 years ago
Depends on: 819091
(Reporter)

Updated

6 years ago
Depends on: 819095
OS: Mac OS X → Gonk (Firefox OS)
Hardware: x86 → ARM
(Reporter)

Comment 18

6 years ago
These are some informal updated startup times I took last week on Jan. 14th build for Otoro.  Unfortunately I don't have numbers for Otoro from October, so I am listing them next to the Unagi numbers from October.

Timing method used was the same for both:

1 Restart device
2 Unlock device
3 Wait 30 seconds
4 Tap icon and start stopwach
5 Wait until App can be interacted with and stop stopwatch (this means the UI has fully rendered and no visible loading is taking place).

The process was done 2 times for each app.

--

Oct. Unagi vs Jan. 14 Otoro Numbers

--

  App Startup Times

  App            Unagi/Oct Otoro/Jan

  Settings     - 4.1       / 3.0 -> 27%
  Browser      - 2.5s      / 1.5 -> 40%
  Camera       - 4.7s      / 5
  Dialer       - 5s        / 2.3 -> 54%
  Contacts     - 5.7       / 3.6 -> 36%
  E-Mail       - 9-11.5    / 5.2 (no accounts setup)
  Cost Control - 12.7s     / 1.8 -> 78%
  Calendar     - 11.3s     / 3.3 -> 70%
  Clock        - 4.8       / 2.7 -> 43%
  SMS          - 3s        / 2.2 -> 27%
  Gallery      - 4s        / 2.7 -> 32%
  Music        - 5s        / 2.8 -> 44%
  Video        - 5s        / 5
  FM Radio     - 3.4s      / 2   -> 41%
  Marketplace  - 4.5-6s    / 1.9 (no connectivity)

  Return to Homescreen

  - 1-3 sec typical, sometimes 3+ / Currently: ~.5s or less

--
Whiteboard: UX-P1 → UX-P1, [perf_tsk]
Whiteboard: UX-P1, [perf_tsk] → UX-P1, [FFOS_perf]

Updated

6 years ago
blocking-b2g: --- → tef?
This is a meta bug and we only block on bugs that are actionable.
blocking-b2g: tef? → ---
Duplicate of this bug: 832097

Updated

6 years ago
blocking-b2g: --- → tef?
Please read comment 19. Don't renom this one. Thanks.
blocking-b2g: tef? → ---

Updated

6 years ago
Whiteboard: UX-P1, [FFOS_perf] → UX-P1, [FFOS_perf], PRODUCT-PERF
Depends on: 836361

Comment 22

6 years ago
cc Arun for additional metrics.

Comment 23

6 years ago
Per comment 19 -- if this bug is not actionable, is there a reason this bug is still open with a NEW status? I think something like this should be marked RESOLVED WONTFIX or INVALID or similar so the bug DB can stay relevant. Or is adding the keyword "meta" enough and people should just know to filter those bugs out if they're looking to contribute?
Hi Darius. This is very common in Mozilla development to have a meta bug like this which identifies a set of individually actionable items of work, and can itself be closed once all of those items are closed.

If you see a meta-bug like this, check out each of the dependent bugs to see which are unowned. Feel free to email me directly if you want more info.
Depends on: 796467
Depends on: 849280
Whiteboard: UX-P1, [FFOS_perf], PRODUCT-PERF → UX-P1, [FFOS_perf], PRODUCT-PERF c=

Updated

5 years ago
Whiteboard: UX-P1, [FFOS_perf], PRODUCT-PERF c= → UX-P1, PRODUCT-PERF [c= ]

Updated

5 years ago
No longer depends on: 819080
FIXED with all the performance work done in the last month.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED

Updated

5 years ago
Whiteboard: UX-P1, PRODUCT-PERF [c= ] → UX-P1, PRODUCT-PERF [c= p= s=2013.10.25 u=]
You need to log in before you can comment on or make changes to this bug.