Closed Bug 970188 Opened 10 years ago Closed 10 years ago

[Buri][Keyboard][Performance] The re-launch time of keyboard is around ~1.2 seconds

Categories

(Firefox OS Graveyard :: Gaia::Keyboard, defect, P3)

ARM
Gonk (Firefox OS)
defect

Tracking

(Not tracked)

RESOLVED FIXED
2.0 S6 (18july)

People

(Reporter: whsu, Assigned: timdream)

References

Details

(Keywords: perf, Whiteboard: [c=progress p= s=2014.07.18.t u=] )

Attachments

(2 files)

* Description:
  After we did the keyboard performance measurement (Bug 946298), we found keyboard re-launch time increases 219.3 centiseconds. We need to make a decision whether we need to improve keyboard re-launch time.
  Attaching the result. (Keyboard_Launch_Time.png)

* Precondition:
  Install a third party keyboard and enable keyboard OOP

* Reproduction steps:
  1. Restart the Firefox OS device
  2. Kill the built-in keyboard process after FxOS is booted up.
  3. Tap the search field of E.ME.

* Expected result:
  The keyboard relaunch time is close to keyboard launch time

* Actual result:
  The re-launch time increases 219.3 centiseconds.

* Reproduction build:( V1.4 )
  - Location: /pvt/mozilla.org/b2g_ril/hamachi-eng-mozilla-central-20140126040203-ril01.02.00.019.102.zip 
  - Build Id: 20140126004002 
  - Gaia: f382061fe95750d584a9078175c421a36892afc9 
  - Gecko: 3f1dd2a8e972
blocking-b2g: --- → 1.4?
Whiteboard: [FT:System-Platform], [3rd-party-keyboard]
This is a perf regression from enabling oop keyboard support.
Blocks: 942790
Keywords: perf, regression
Note - the bug specified here is indicating that the oop keyboard on average was 2.193 seconds longer on keyboard re-launch than the non-oop keyboard.
Priority: -- → P1
Whiteboard: [FT:System-Platform], [3rd-party-keyboard] → [c=effect p= s= u=] [FT:System-Platform], [3rd-party-keyboard]
This is not a regression since we are measuring keyboard re-launch time after OOM.

Keyboard was never engineered to be relaunched, so some feature work needed to get this right. Obviously cold launch speed can never be as fast as warn launch, so I would say if we target getting keyboard back in 1 sec, it's acceptable.

Triage: +'ing to find a way to reach that target launch time.

Rudy, you should be looking at this bug already right? Any update?
Assignee: nobody → rlu
blocking-b2g: 1.4? → 1.4+
Flags: needinfo?(rlu)
Keywords: perf, regression
Keywords: perf
Status: NEW → ASSIGNED
Whiteboard: [c=effect p= s= u=] [FT:System-Platform], [3rd-party-keyboard] → [c=effect p= s= u=1.4] [FT:System-Platform], [3rd-party-keyboard]
(In reply to Tim Guan-tin Chien [:timdream] (MoCo-TPE) (please ni?) from comment #3)
> This is not a regression since we are measuring keyboard re-launch time
> after OOM.

Technically this would be a behavioral regression actually, because from an end-user's perspective, this would appear slower than 1.3. On that regard, it's still classified under the regression keyword, even though OOP support only came into the playing field in 1.4.
Keywords: regression
Will look into this issue, and hope to improve the launch in sprint 2.
Thanks.
Flags: needinfo?(rlu)
Target Milestone: --- → 1.4 S2 (28feb)
No longer blocks: 942790
Blocks: 964670
3rd party keyboard will be off after the feature de-scope. Will address this in v1.5 when 3rd party keyboard is pref-on
blocking-b2g: 1.4+ → 1.5?
Target Milestone: 1.4 S2 (28feb) → ---
Priority: P1 → P2
Whiteboard: [c=effect p= s= u=1.4] [FT:System-Platform], [3rd-party-keyboard] → [c=effect p= s= u=] [FT:System-Platform], [3rd-party-keyboard]
Triage: keep 1.5? because we need to make sure we don't ship 1.5 (with keyboard oop enabled) with this bug.

The fix should be either ship keyboard refactor work (bug 956169) or address this bug in the current code base.
blocking-b2g: 1.5? → 1.5+
Priority: P2 → P1
Whiteboard: [c=effect p= s= u=] [FT:System-Platform], [3rd-party-keyboard] → [c=effect p= s= u=1.5] [FT:System-Platform], [3rd-party-keyboard]
Assignee: rlu → timdream
Summary: [Keyboard][V1.4][Performance] The re-launch time of keyboard increases 219.3 centiseconds → [Keyboard][Performance] The re-launch time of keyboard is 2.193 centiseconds
Probably also a lot better once bug 992346 gets landed.
Summary: [Keyboard][Performance] The re-launch time of keyboard is 2.193 centiseconds → [Keyboard][Performance] The re-launch time of keyboard increases 2.193 seconds
Whiteboard: [c=effect p= s= u=1.5] [FT:System-Platform], [3rd-party-keyboard] → [c=effect p= s= u=2.0] [FT:System-Platform], [3rd-party-keyboard]
Attached patch Patch for logSplinter Review
So I just redo the test, and it looks like from various improvement, we are now looking at around 1.2sec of recovery time.

==

1397632390851 keyboard_manager.js:inputmethod-contextchange
1397632390896 keyboard_manager.js:setLayoutFrameActive:false
1397632390906 keyboard_manager.js:setLayoutFrameActive:true
1397632391576 keyboard.js
1397632391802 keyboard.js:initKeyboard
1397632391811 keyboard.js:loadLayout
1397632391829 keyboard.js:loadLayout:loaded
1397632392014 keyboard.js:updateTargetWindowHeight

1397632392014 - 1397632390851 = 1163 ms
1397632391576 - 1397632390851 = 725 ms

==

E/GeckoConsole( 1127): Content JS LOG at app://system.gaiamobile.org/js/keyboard_manager.js:153 in km_init/<: 1397632786529 keyboard_manager.js:inputmethod-contextchange
E/GeckoConsole( 1127): Content JS LOG at app://system.gaiamobile.org/js/keyboard_manager.js:389 in km_loadKeyboardLayout: === Enable keyboard: app://keyboard.gaiamobile.org run as OOP ===
E/GeckoConsole( 1127): Content JS LOG at app://system.gaiamobile.org/js/keyboard_manager.js:777 in km_setLayoutFrameActive: 1397632786574 keyboard_manager.js:setLayoutFrameActive:false
E/GeckoConsole( 1127): Content JS LOG at app://system.gaiamobile.org/js/keyboard_manager.js:777 in km_setLayoutFrameActive: 1397632786584 keyboard_manager.js:setLayoutFrameActive:true
E/GeckoConsole( 1474): Content JS LOG at app://keyboard.gaiamobile.org/js/keyboard.js:4 in anonymous: 1397632787238 keyboard.js
E/GeckoConsole( 1474): Content JS LOG at app://keyboard.gaiamobile.org/js/keyboard.js:305 in initKeyboard: 1397632787470 keyboard.js:initKeyboard
E/GeckoConsole( 1474): Content JS LOG at app://keyboard.gaiamobile.org/js/keyboard.js:482 in loadLayout: 1397632787479 keyboard.js:loadLayout
E/GeckoConsole( 1474): Content JS LOG at app://keyboard.gaiamobile.org/js/keyboard.js:488 in setKeyboardName/loadLayout/script.onload: 1397632787496 keyboard.js:loadLayout:loaded
E/GeckoConsole( 1474): Content JS LOG at app://keyboard.gaiamobile.org/js/keyboard.js:889 in updateTargetWindowHeight: 1397632787683 keyboard.js:updateTargetWindowHeight

1397632787683 - 1397632786529 = 1154 ms

==

I will be working on various patches to improve the launch time of the keyboard app.
Converting this to a meta bug. Not removing blocking status for now until I have dependent bug filed.
Summary: [Keyboard][Performance] The re-launch time of keyboard increases 2.193 seconds → [meta][Keyboard][Performance] The re-launch time of keyboard is around ~1.2 seconds
I think with all patches from https://bugzilla.mozilla.org/show_bug.cgi?id=994000 we will get it down to 700 ms. or so.
Adding story points first.
Whiteboard: [c=effect p= s= u=2.0] [FT:System-Platform], [3rd-party-keyboard] → [FT:System-Platform], [3rd-party-keyboard][p=3]
Whiteboard: [FT:System-Platform], [3rd-party-keyboard][p=3] → [FT:System-Platform], [3rd-party-keyboard][p=3][c=effect p= s= u=2.0]
Make this officially a meta bug.
blocking-b2g: 2.0+ → ---
Depends on: 994000
Keywords: perf
Whiteboard: [FT:System-Platform], [3rd-party-keyboard][p=3][c=effect p= s= u=2.0]
A readable version (w/ time between logs) of comment 9:


keyboard_manager.js:153 in km_init/<: 
1397632786529 keyboard_manager.js:inputmethod-contextchange

45ms

keyboard_manager.js:777 in km_setLayoutFrameActive: 
1397632786574 keyboard_manager.js:setLayoutFrameActive:false

10ms (bug 1005752)

keyboard_manager.js:777 in km_setLayoutFrameActive: 
1397632786584 keyboard_manager.js:setLayoutFrameActive:true

654ms ?

keyboard.js:4 in anonymous: 
1397632787238 keyboard.js

232ms getSettings() (bug 1005751)

keyboard.js:305 in initKeyboard: 
1397632787470 keyboard.js:initKeyboard

9ms

keyboard.js:482 in loadLayout: 
1397632787479 keyboard.js:loadLayout

17ms (bug 1005753)

keyboard.js:488 in setKeyboardName/loadLayout/script.onload: 
1397632787496 keyboard.js:loadLayout:loaded

187ms ?

keyboard.js:889 in updateTargetWindowHeight: 
1397632787683 keyboard.js:updateTargetWindowHeight
(In reply to Tim Guan-tin Chien [:timdream] (MoCo-TPE) (please ni?) from comment #13)
> Make this officially a meta bug.

The keyword & the blocking flag should not be removed here. Performance bugs are filed to indicate that a certain goal is being missed for a target metric (this is definition of the bug from an end-user perspective). How it's resolved is up to the developer based on the study of the problems seen, which could be fixed in the actual bug itself or the issues found during the study of the bug itself. This typically results in both the bug describing symptom being a blocker along with any other issues filed separately that are required as blockers. But it's not tracked as a meta per say.
blocking-b2g: --- → 2.0+
Keywords: perf
Summary: [meta][Keyboard][Performance] The re-launch time of keyboard is around ~1.2 seconds → [Keyboard][Performance] The re-launch time of keyboard is around ~1.2 seconds
(In reply to Jason Smith [:jsmith] from comment #15)
> The keyword & the blocking flag should not be removed here. Performance bugs
> are filed to indicate that a certain goal is being missed for a target
> metric (this is definition of the bug from an end-user perspective). How
> it's resolved is up to the developer based on the study of the problems
> seen, which could be fixed in the actual bug itself or the issues found
> during the study of the bug itself. This typically results in both the bug
> describing symptom being a blocker along with any other issues filed
> separately that are required as blockers. But it's not tracked as a meta per
> say.

Yeah, make sense. So how can I call this bug fixed if everything is fixed on dependent bug?
No longer blocks: 1005764
Depends on: 1005764
(In reply to Tim Guan-tin Chien [:timdream] (MoCo-TPE) (please ni?) from comment #14)
> keyboard.js:488 in setKeyboardName/loadLayout/script.onload: 
> 1397632787496 keyboard.js:loadLayout:loaded
> 
> 187ms ?
> 
> keyboard.js:889 in updateTargetWindowHeight: 
> 1397632787683 keyboard.js:updateTargetWindowHeight

This is bug 1005764
(In reply to Tim Guan-tin Chien [:timdream] (MoCo-TPE) (please ni?) from comment #16)
> (In reply to Jason Smith [:jsmith] from comment #15)
> > The keyword & the blocking flag should not be removed here. Performance bugs
> > are filed to indicate that a certain goal is being missed for a target
> > metric (this is definition of the bug from an end-user perspective). How
> > it's resolved is up to the developer based on the study of the problems
> > seen, which could be fixed in the actual bug itself or the issues found
> > during the study of the bug itself. This typically results in both the bug
> > describing symptom being a blocker along with any other issues filed
> > separately that are required as blockers. But it's not tracked as a meta per
> > say.
> 
> Yeah, make sense. So how can I call this bug fixed if everything is fixed on
> dependent bug?

I'd call it fixed if we resolve enough issues to cause us to hit the goal again, which will require a certain set of dependencies to be fixed (not necessarily all of them though, as you might fix a couple and hit the goal).
Whiteboard: [c=progress p= s= u=2.0]
Please see bug 1013155 comment 5 on latest measurement. The original measurement with log is a bit of and did not consider layout rendering time.
Hi, all,

Thanks for your fully support.

I used Buri and OpenC to re-test keyboard performance.
Here are the results. FYI.

# Buri (V2.0)			
 *E.ME:				
  1. 2.63		
  2. 2.64		
  3. 2.83		
  ~ Average: 2.7 seconds					
					
 * Message:				
  1. 1.96		
  2. 2.46		
  3. 2.8
  ~ Average: 2.40 seconds		
					
 * Browser				
  1. 2.64		
  2. 2.33		
  3. 2.33		
  ~ Average: 2.43 seconds				
					
					
# OpenC (V2.0)
PS: I don't know why the OpenC can show keyboard animation.
    So, I provide two timestamps for your reference.
    The first one indicate that mean time to show cursor till keyboard animation appeared.
    The second one indicate that mean time to complete keyboard animation.
 * E.ME
  1. 0.90    0.16
  2. 0.76    0.20
  3. 0.76    0.24
  ~  0.80    0.20 (Average)
			
 * Message:			
  1. 1.17    0.33
  2. 1.27    0.30
  3. 1.00    0.30
  ~  1.14    0.31 (Average)

 * Browser				
  1. 1.50    0.24
  2. 1.40    0.27
  3. 1.16    0.30
  ~  1.35    0.27 (Average)


# Comment:
 1. If we follow the acceptance criteria that performance team defined (1250 ms, flame), I think OpenC can achieve the goal.
    ~ https://wiki.mozilla.org/FirefoxOS/Performance/Release_Acceptance
 2. We doesn't seem to support Buri on V2.0 because QA has stopped tests on Buri with 2.0 build.
    So, These data collected from Buri may be inappropriate.
 3. Buri seem to have graphic issue. Because OpenC can show keyboard animation, but Buri can't do that.
Hum... I am a bit surprised the keyboard takes longer to show up in the Browser app than Message for Open C. Anyhow, these cold launch timing means we should no longer block on this bug (Not the that the user will encounter cold launch frequently on these phone anyway).

Jason, you have been watching this feature closely. Do you support the argument here?

I will keep this bug open and close once I have all my launch time performance improvement landed.
blocking-b2g: 2.0+ → 2.0?
Flags: needinfo?(jsmith)
Hi, all,

I record a demo video for your reference.
- https://www.youtube.com/watch?v=7M_OIFZoMP4

Keyboard has notable improvement on V2.0.
Thanks!
The one question I have is - are there any devices we need to support in 2.0 timeframe that can demonstrate slower performance than the Open C? My hunch is that answer to this question is no, but we should probably confirm that to say it's okay to accept the Open C performance numbers. If the answer ends up being no, then we can unblock this bug.

Let me email Kevin about this & cc you guys.
Flags: needinfo?(jsmith)
Got my answer offline - it's no, so I'm removing the nom here.
blocking-b2g: 2.0? → ---
Keywords: regression
Whiteboard: [c=progress p= s= u=2.0] → [c=progress p= s= u=]
Dropping to P3 per comment 20 and leaving for Tim to close per comment 21.

FxOS Performance Triage
Priority: P1 → P3
Summary: [Keyboard][Performance] The re-launch time of keyboard is around ~1.2 seconds → [Buri][Keyboard][Performance] The re-launch time of keyboard is around ~1.2 seconds
With bug 1039171 preload keyboard app will run inproc for devices of memory below 512MB. Devices with more memory does not affected by this bug. I am calling this bug as fixed.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Whiteboard: [c=progress p= s= u=] → [c=progress p= s=2014.07.18.t u=]
Target Milestone: --- → 2.0 S6 (18july)
You need to log in before you can comment on or make changes to this bug.