1035506 - [B2G][Window Management] Task Manager is slow to load or doesnt load at all

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::System::Window Mgmt, defect, P1)

Description

[Josh Schmitt [Joshs]]

Attachment: log.txt

Description:
While in Apps or on the Homescreen, the Task Manager is extremely slow to load or does not load at all.

Repro Steps:
1) Update a Flame to 20140707000200
2) Open Gallery app
3) Long press the 'Home' button

Actual:
The Task Manager loads slowly or doesnt load at all.

Expected:
The Task Manager to always load and load quickly.

Environmental Variables:
Device: Flame 2.0
Build ID: 20140707000200
Gaia: ef67af27dff3130d41a9139d6ae7ed640c34d922
Gecko: f53099796238
Version: 32.0a2 (2.0)
Firmware Version: v122

User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0

Notes:
Repro frequency: 100%,
See attached: logcat

Comment 1

[Peter Bylenga [:PBylenga]]

Adding qawanted for flame branch checks with memory set to 273mb.

Comment 2

[Cody Roesch [:croesch]]

This bug repro's on: Flame 2.1 Master, Flame 2.0, Flame 1.4

Actual Results: With Mem set to 273mb, the Task Manager does take a long time to appear and is problematic.

**Flame 1.4 is not nearly as bad as 2.1 and 2.0 but the task manager can still take 3-4 seconds on occassion (3/8) attempts so i listed it as affected.

Environmental Variables:
Device: Flame Master
Build ID: 20140708061447
Gaia: 740faa5d0060fb218b407cf224330654ddf833a5
Gecko: 8bfe3372f848
Version: 33.0a1 (Master)
Firmware Version: v122
-----------------------------------------------
Environmental Variables:
Device: Flame 2.0
Build ID: 20140708071430
Gaia: 444efc47370736a7bc57cc72a1ec00f3cc5f7e92
Gecko: c3683c06f6d8
Version: 32.0a2 (2.0)
Firmware Version: v122
-----------------------------------------------
Environmental Variables:
Device: Flame 1.4
Build ID: 20140708070432
Gaia: 229864ff4ad90899f017341b9e81ed0b53498c66
Gecko: 7e146a1d7542
Version: 30.0 (1.4)
Firmware Version: v122

Comment 3

[Jason Smith [:jsmith]]

Josh - If the reproduction rate gets considerably worse between releases, then the bug is classified as a regression. The above comments imply:

* 1.4 - 3/8 reproduction rate
* 2.0 - 100% reproduction rate

Which implies a regression between releases.

"perf" should be flagged in the keywords as well, since this is a performance issue.

Comment 4

[Joshua Mitchell (Inactive)]

ahhh.... ya. Understood.

Comment 5

[Jason Smith [:jsmith]]

And you need to triage this for a blocking decision as well.

Comment 6

[Joshua Mitchell (Inactive)]

nomming 2.0 as this is a regression and a performance issue

Comment 7

[Jon Hylands [:jhylands]]

Once QA has come back with a reasonable regression window, I'll bisect to find the specific commit, and assign as appropriate.

Comment 8

[Andreas Gal :gal]

This is likely the new homescreen so not sure I would wait for a regression window here. How about a memory profile for starters and maybe a CPU profile?

Comment 9

[Gregor Wagner [:gwagner]]

Might be related to loading screen-shots. Maybe we should switch to icons for 256 MB devices like we did in tarako?

Comment 10

[Jason Smith [:jsmith]]

(In reply to Andreas Gal :gal from comment #8)
> This is likely the new homescreen so not sure I would wait for a regression
> window here. How about a memory profile for starters and maybe a CPU profile?

Okay. Switching to qawanted first to get an about:memory report.

(In reply to Gregor Wagner [:gwagner] from comment #9)
> Might be related to loading screen-shots. Maybe we should switch to icons
> for 256 MB devices like we did in tarako?

Note - we never needed to do that for other devices that had 256 MB support in past releases. Why would we need this now when we didn't need it before?

Comment 11

[Cody Roesch [:croesch]]

Attachment: about-memory-3.zip

This Get about Mem report is from the following build:

Environmental Variables:
Device: Flame Master
Build ID: 20140710071153
Gaia: 09642e74e250fbc62db860c808ef188628fca55d
Gecko: f93c0ef45597
Version: 33.0a1 (Master)
Firmware Version: v122

Comment 12

[Joshua Mitchell (Inactive)]

holding off on re-adding the regression-window-wanted until the memory-report gets a look-over.

Comment 13

[Bruce Huang [:bhuang] (inactive)]

(In reply to Jason Smith [:jsmith] from comment #10) 
> Note - we never needed to do that for other devices that had 256 MB support
> in past releases. Why would we need this now when we didn't need it before?

Previous 256 MB devices only ran at HVGA resolutions.  With FWVGA on Flame the remaining memory is going to be less compared to those devices.

Comment 14

[Jason Smith [:jsmith]]

(In reply to Bruce Huang [:bhuang] <[e-mail]> from comment #13)
> (In reply to Jason Smith [:jsmith] from comment #10) 
> > Note - we never needed to do that for other devices that had 256 MB support
> > in past releases. Why would we need this now when we didn't need it before?
> 
> Previous 256 MB devices only ran at HVGA resolutions.  With FWVGA on Flame
> the remaining memory is going to be less compared to those devices.

Right, but I thought that's why we tried to account for this when we decided to do testing by default on 273 MB, not 256 MB? Or do we need to adjust the default here differently here to account for FWVGA?

Comment 15

[Gregor Wagner [:gwagner]]

(In reply to Jon Hylands [:jhylands] from comment #7)
> Once QA has come back with a reasonable regression window, I'll bisect to
> find the specific commit, and assign as appropriate.

Any updates here?

Comment 16

[Jon Hylands [:jhylands]]

Kyle, can you point someone at the memory report attached, and provide some comment?

Comment 17

[Jon Hylands [:jhylands]]

Although it seems to me like this would be easier to analyze if we had a before and after memory report...

Comment 18

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

The things I would take away from this report.

1) The heap-unclassified in the parent process is a little high, but not extremely slow.  It's 21.52%, and 15-20% would be more normal.
2) The system app is using a *lot* of DOM memory.  On my freshly started Nexus 5 it's using 0.44 MB with only 0.14 MB of text-nodes.  In this report it's using 0.9 MB of text-nodes.  There are several hundred text nodes that are children of the <head> element in the system app which seems quite unusual.  We should probably spin off another bug for that.
3) The Settings app is still running, and it should get OOM killed if we really need the memory.

Based on this I don't think there's a "smoking gun" in the memory report.  The next step is probably a CPU profile.

Comment 19

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Although those might all be text nodes for the <script> and <link> tags.  I'll investigate further.

Comment 20

[Jon Hylands [:jhylands]]

Joshua, can you re-add the regression-window-wanted (as noted in Comment 12) so we can figure out where the problem is?

Comment 21

[Candice Serran (:cserran)]

Mike can you help us get a regression window on this so we can start working on this blocker?

Comment 22

[Jon Hylands [:jhylands]]

I started looking at this, and I realized we don't have any solid timing numbers. In order to bisect, I need to know what the bad-case is, and what the good case is.

Comment 23

[Jon Hylands [:jhylands]]

http://youtu.be/hbSg15VCBIs

This is a video showing a long press after launching gallery. From the time my finger touches the button to the time the task manager shows up is almost exactly two seconds. One of those seconds is waiting for long press, which means it takes the task manager one second to launch.

This is on a flame running latest from master (2.1), booted in 273MB mode.

Comment 24

[Sam Foster [:sfoster] (he/him)]

I'm trying to reproduce on my 273MB Flame and get my own numbers and I'm not able to get it any like as smooth as your video. Also, in practice every app is immediately killed when it goes to the background, task manager is not very useful in this configuration; you need to toggle the Developer -> Window Management -> App Suspending setting to get anything more than the current app in the task switcher. Interestingly, this seems to greatly improve the task manager responsiveness

Comment 25

[Mike Lee [:mlee]]

Candice, Jon's already helping here. Per his comment 22, he needs more information to do any reasonable bisection to find the regression point.

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #18)
> ...
> There are several hundred text nodes that
> are children of the <head> element in the system app which seems quite
> unusual.  We should probably spin off another bug for that.

Kyle, have you filed a bug for this? If so please link to it here.

> 3) The Settings app is still running, and it should get OOM killed if we
> really need the memory.

Do we need a bug for this issue as well, or are we already tracking this as part of other LMK fixes?

> Based on this I don't think there's a "smoking gun" in the memory report. 
> The next step is probably a CPU profile.

Jon, please capture a CPU profile per Kyle's suggestion.

Comment 26

[Joshua Mitchell (Inactive)]

(In reply to Jon Hylands [:jhylands] from comment #20)
> Joshua, can you re-add the regression-window-wanted (as noted in Comment 12)
> so we can figure out where the problem is?

Only as an absolute last resort. We do not usually request regression-windows on issues that do not have a clear "broken" and "fixed" state. It is very difficult for a tester to narrow down a window based on variable data - as the prior comments are discussing. We also do not request windows with such low repro rates (3/8 on the 1.4 branch). Getting a window on that is problematic at best.

Comment 27

[Jon Hylands [:jhylands]]

Sam, perhaps it would be more useful for us to produce a CPU profile on your phone, since I don't seem to have any luck reproducing this behavior on mine. Do you know how to produce a CPU profile? If not, we can do a vidyo session and I can walk you through it...

Comment 28

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

(In reply to Mike Lee [:mlee] from comment #25)
> Candice, Jon's already helping here. Per his comment 22, he needs more
> information to do any reasonable bisection to find the regression point.
> 
> (In reply to Kyle Huey [:khuey] ([e-mail]) from comment #18)
> > ...
> > There are several hundred text nodes that
> > are children of the <head> element in the system app which seems quite
> > unusual.  We should probably spin off another bug for that.
> 
> Kyle, have you filed a bug for this? If so please link to it here.

I have not.  I need bug 1040380 to investigate this further.

Comment 29

[Sam Foster [:sfoster] (he/him)]

Jon, no I've not produced a CPU profile before. I'm travelling all day today. I'll read up and give it a go on Monday - and reach out for help if necessary.

Comment 30

[Jon Hylands [:jhylands]]

Sam, just so you know, I'll be in Paris all next week (work week), so we'll have to try and sync up in the morning your time if we do need to.

For profiling instructions, see: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler

Comment 31

[Ross Kuhlman ([e-mail])]

Task manager performance is improved in the current build due to changes in bug 1029902.

Comment 32

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

If we could get another log like comment 11 from a build with the patch from bug 1040380 in it I could try to figure out where the extra text node usage is coming from.

Comment 33

[Jason Smith [:jsmith]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #32)
> If we could get another log like comment 11 from a build with the patch from
> bug 1040380 in it I could try to figure out where the extra text node usage
> is coming from.

Sure. Marking qawanted to get another about:memory report with the latest 2.0 build.

Comment 34

[Jason Smith [:jsmith]]

(In reply to Jason Smith [:jsmith] from comment #33)
> (In reply to Kyle Huey [:khuey] ([e-mail]) from comment #32)
> > If we could get another log like comment 11 from a build with the patch from
> > bug 1040380 in it I could try to figure out where the extra text node usage
> > is coming from.
> 
> Sure. Marking qawanted to get another about:memory report with the latest
> 2.0 build.

Also including qawanted to retest this on 319 MB Flame.

Comment 35

[Peter Bylenga [:PBylenga]]

[Blocking Requested - why for this release]:
Renominating to not block release given that at 319MB task manager appears as expected.  At 273MB it is still choppy during the transition.

Current state at 319MB does not block smoke test.

Environmental Variables:
Device: Flame 2.0
Build ID: 20140724000201
Gaia: 29266e18c35f4e72e35f1bba0e34f2fb6b995cc3
Gecko: 178fe2efc41d
Version: 32.0 (2.0)
Firmware Version: v122
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0

Comment 36

[Peter Bylenga [:PBylenga]]

removing qawanted as it was answered in Comment 35

Comment 37

[Jon Hylands [:jhylands]]

We're going to close this as wont-fix.