856524 - Gaia apps freezes during stability tests

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: General, defect, P2)

Description

[Pooja]

User Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111107 Ubuntu/10.04 (lucid) Firefox/3.6.24
Build ID: 20111107172717




Expected results:

After running few tests for a day or two , device  UI freezes i.e either
1. All the icons disappeared from the homescreen. Or
2. Homescreen shows all the icon but unable to switch/open any other app (even volume key also doesn't work).Or
3. Particular app freeze eg: Dialer app. where rest all apps works normally
 
Test that we played:
1.Played Music in background,
2.Enable BT,
3.Receive multiple Mobile terminated (MT) calls,

Comment 1

[Pooja]

Logs at :oom ,procrank and adb logcat
https://www.dropbox.com/sh/8nlqewu0uygw8dh/ejiTcL2Odl

b2g-ps --oom indicates that as soon as device runs for long time the main process started consuming 90MB and more RSS. Sometimes its reaches upto 109MB too.

Comment 2

[Michael Vines [:m1] [:evilmachines]]

(leo+ until we get to the root cause of the main process bloat during long term stability test)

Comment 3

[Pooja]

Also would like to add here, Most of the time we have observed that  UI freezes only when we have MT call test case in our test suite.
So possibly dialer app could be the reason for these freezes.

Similarly,various UI freezes was also noticed by jgriffin in below bug
https://bugzilla.mozilla.org/show_bug.cgi?id=841976 (just FYI)

Comment 4

[Kai-Chih Hu [:khu]]

Do we know if it reached OOM?

Comment 5

[Pooja]

HI Khu,

From OOM logs its seems like device is going to low memory. 
b2g                  0        454         0          root      131   1     214456 90996 ffffffff 400ff430 S /system/b2g/b2g

After some time b2g process started bloating and kill few app  and we have seen continuous hang in device.After few more hours all the home screen icon disappeared completely and it get freezed and it also looses its all network connection.
Let me know if you need any further information.

Comment 6

[Mike Habicher [:mikeh] (high bugzilla latency)]

Possibly related to bug 846903, which is also showing long-term process bloat.

Comment 7

[Kai-Chih Hu [:khu]]

Ahuang, is this something that you can help? Thanks!

Comment 8

[Alan Huang]

Besides ps, procrank, logcat and top, this bug also needs "about-memory" (from tools/get_about_memory.py) and:
1. procmem of /system/b2g/b2g
2. showmap of /system/b2g/b2g
3. librank
4. kernel log (dmesg)

Thanks!

Comment 9

[Pooja]

HI Huang,

We have started test to collect about-memory logs with DMD enabled.
Will update with all the required logs shortly.

Comment 10

[Pooja]

HI Alan Hung,

I have asked tester to reproduce the issue but device got crashed due to some other issues.I will update this bug as soon as we get the logs with UI freeze

Thanks,
Pooja

Comment 11

[Pooja]

HI Hung,
Please find all the required logs at :

https://www.dropbox.com/sh/vb84atiirm8dv3l/caCxBSNq_T?m

Device UI was freezed i.e no icon no alerts at homescreen even though we were able to receive the calls (auto answer was enabled)and music app was also running in background.

Volume button also stooped working i.e not able to increase or decrease vol.

Comment 12

[Pooja]

HI,
Is there any update on this?

Comment 13

[Gabriele Svelto [:gsvelto] (PTO)]

I've just fixed a memory leak that affects the main B2G process as part of bug 864516. Long story short when the lockscreen shows up we fired a timer using setTimeout() that would keep itself alive by calling setTimeout() again. Since this timer was never cleared and two new ones would be created every time the screen was locked/unlocked these would start to pile up causing two major symptoms: memory slowly growing in the main process and increasingly expensive CPU spikes at every minute mark. The homescreen was affected by a similar problem but being a regular application it would eventually be killed because of the OOM condition and then restarted.

I would encourage all long-running gaia tests doing a large number of lock/unlock transitions to be re-run now that the fixes for bug 864516 have landed.

Comment 14

[Pooja]

Thanks Gabriele.
We will verify this issue again on latest build and will update you about results.

Comment 15

[Wayne Chang [:wchang]]

Hi PoojaS, When can we expect to see updated results from the latest build? Would just like to confirm that the problem is no longer seen to solve this bug.

Comment 16

[Michael Vines [:m1] [:evilmachines]]

I'll probably take us several dates without seeing this issue re-occur with the fix in bug 864516 applied to give us enough confidence to WFM this bug.  We'll be picking up bug 864516 in our v1.1 stability test tomorrow

Comment 17

[Gabriele Svelto [:gsvelto] (PTO)]

I've got a fix for bug 864516 ready for v1.0.1 too but it's not been merged yet. Do you need that one applied too for your testing?

Comment 18

[Pooja]

Thanks Gabriele and Chang
we  tried on latest build and the issue is no more re-producible even after long test run.
Seems its resolved ,we can close this bug now.

Comment 19

[Gabriele Svelto [:gsvelto] (PTO)]

(In reply to PoojaS from comment #18)
> Thanks Gabriele and Chang
> we  tried on latest build and the issue is no more re-producible even after
> long test run.
> Seems its resolved ,we can close this bug now.

Thanks for testing this, closing as WORKSFORME.