1119608 - [SMS][dolphin][FFOS7715 v2.1][crash] Sms crashed during monkey test.

Status: RESOLVED FIXED

(Firefox OS Graveyard :: RIL, defect)

Description

[Wei Gao (Spreadtrum)]

OS version
---------------------------------------------
FireFoxOS v2.1

Reproduce steps:
---------------------------------------------
monkey test.

Actual result:
---------------------------------------------
Sms crashed.

Probability:
---------------------------------------------
Occasionally Recurrence

Comment 1

[Wei Gao (Spreadtrum)]

Attachment: crash.txt

There is URL=app://sms.gaiamobile.org/manifest.webapp
in file "mozilla/crashreorters/pending/*.extra："

Comment 2

[Oleg Zasypkin [:azasypkin]]

Don't really know where to start with this issue :)

Wei, could you please provide clearer STR and hardware conditions (RAM and etc.) so that we can try to reproduce it on our end?

Thanks!

Comment 3

[Wei Gao (Spreadtrum)]

(In reply to Oleg Zasypkin [:azasypkin] from comment #2)
> Don't really know where to start with this issue :)
> 
> Wei, could you please provide clearer STR and hardware conditions (RAM and
> etc.) so that we can try to reproduce it on our end?
> 
> Thanks!

Hi Oleg

We run monkey test, so I also have no clearly STR.
The hardware is dolphin sp7715ea, its RAM is 512MB.
$ ./check_versions.sh 
Gaia-Rev        73be51f998031f06db0cd660c0e388fa621c9f4c
Gecko-Rev       30de9395e2c3e4c3d640bc6c70ddbc1a8c8cf88f
Build-ID        20150105041406
Version         34.0
Device-Name     scx15_sp7715ea
FW-Release      4.4.2
FW-Incremental  53
FW-Date         Mon Jan  5 04:08:52 CST 2015

If any information is needed, I will provide.
Thanks.

Comment 4

[Wei Gao (Spreadtrum)]

Attachment: backtrace-2.txt

We reproduced this issue again today.

The attachment is the backtrace.
Thanks.

Comment 5

[Julien Wajsberg [:julienw]]

Hey Bevis, looks like a crash in MobileMessage, maybe you'll be able to follow-up with the right engineers?

Comment 6

[Bevis Tseng[:bevis][:btseng](Exited)]

I take this bug first to follow up.

Comment 7

[Bevis Tseng[:bevis][:btseng](Exited)]

(In reply to Wei Gao (Spreadtrum) from comment #3)
> We run monkey test, so I also have no clearly STR.
> The hardware is dolphin sp7715ea, its RAM is 512MB.
> $ ./check_versions.sh 
> Gaia-Rev        73be51f998031f06db0cd660c0e388fa621c9f4c
> Gecko-Rev       30de9395e2c3e4c3d640bc6c70ddbc1a8c8cf88f
> Build-ID        20150105041406
> Version         34.0
> Device-Name     scx15_sp7715ea
> FW-Release      4.4.2
> FW-Incremental  53
> FW-Date         Mon Jan  5 04:08:52 CST 2015
> 
> If any information is needed, I will provide.
> Thanks.

Hi,

Would you mind telling us 
1. The steps of starting the monkey test?
   I'd like to see if we can reproduce this locally to figure out the root cause for 
   further verification when fixing it.
2. What's the precisely reproduced rate?
   - How many times did you test?
   - How many times was the sms app crashed?
3. How long does it take from the start time of testing to the time the sms app is crashed?

Thanks!

Comment 8

[Bevis Tseng[:bevis][:btseng](Exited)]

NI for questions in comment 7.

Comment 9

[Vance Chen [:vchen][skype:vance.lucida][[e-mail]]]

Hi Wei Gao -

Since we have some 7715ea devices here in Taipei, it would be nice if you can provide your monkey script such that we can run the Monkey test here to observe the symptoms and capture the logs we need. Thanks!

Comment 10

[Wei Gao (Spreadtrum)]

Attachment: MyMonkey_Test.zip

(In reply to Bevis Tseng[:bevistseng][:btseng] from comment #7)

Hi Bevis
Thanks so much for your attention.

> 1. The steps of starting the monkey test?
>    I'd like to see if we can reproduce this locally to figure out the root
> cause for 
>    further verification when fixing it.

I have uploaded our monkey test script.
We executed MyMonkey_Test/test-config/run-7715ea_v2.1-hudson.sh

> 2. What's the precisely reproduced rate?
>    - How many times did you test?
>    - How many times was the sms app crashed?

We use 16 phones to run the monkey test 10~12 hours every day, and there will be at least one phone to reproduce this issue. 

> 3. How long does it take from the start time of testing to the time the sms
> app is crashed?

I have exported the log. The crash report shows:
CrashTime=1420654568
StartupTime=1420649174
ProcessType=
URL=app://sms.gaiamobile.org/manifest.webapp

But unfortunately I can't understand the time format "1420654568".

Comment 11

[Wei Gao (Spreadtrum)]

Attachment: CrashReport.zip

(In reply to Vance Chen [:vchen][[e-mail]] from comment #9)
> Hi Wei Gao -
> 
> Since we have some 7715ea devices here in Taipei, it would be nice if you
> can provide your monkey script such that we can run the Monkey test here to
> observe the symptoms and capture the logs we need. Thanks!

Dear Vance

Thanks for your reply.
I want to upload the slog as well, but it is too large.
I can only upload the crash report.
If the slog is needed, please let me know.

Comment 12

[Vance Chen [:vchen][skype:vance.lucida][[e-mail]]]

(In reply to Wei Gao (Spreadtrum) from comment #11)
> Created attachment 8549335 [details]
> CrashReport.zip
> 
> (In reply to Vance Chen [:vchen][[e-mail]] from comment #9)
> > Hi Wei Gao -
> > 
> > Since we have some 7715ea devices here in Taipei, it would be nice if you
> > can provide your monkey script such that we can run the Monkey test here to
> > observe the symptoms and capture the logs we need. Thanks!
> 
> Dear Vance
> 
> Thanks for your reply.
> I want to upload the slog as well, but it is too large.
> I can only upload the crash report.
> If the slog is needed, please let me know.

Yes please, the crash log, ram dump and symbol files are quite helpful on checking crash issues. Also to answer your question:

The StartupTime and CrashTime are of Unix Time format:

http://en.wikipedia.org/wiki/Unix_time

So convert to human readable time:

StartupTime = 1420649174 = Wed, 07 Jan 2015 16:46:14 GMT


CrashTime = 1420654568 = Wed, 07 Jan 2015 18:16:08 GMT

So looks like the device crashes after 2 hours

Comment 13

[Wei Gao (Spreadtrum)]

(In reply to Vance Chen [:vchen][[e-mail]] from comment #12)
> So looks like the device crashes after 2 hours

Thanks for your explaintion, that's so kind.

I upload the slog, could you download it?
http://pan.baidu.com/s/1eQAdmoU
If it is unavailable, please let me know.
Thanks.

Comment 14

[Bevis Tseng[:bevis][:btseng](Exited)]

After reviewing the relationship of MobileMessageManager & its parent DOMEventTargetHelper, 
1. APIs that returns DOMRequest/DOMCursor requires valid reference of it's owner window.
2. owner window can be retrieved from DOMEventTargetHelper::GetOwner() which contains a raw pointer of nsPIDOMWindow.
3. DOMEventTargerHelper will be disconnected from its owner window when DOMEventTargetHelper::DisconnectFromOwner() is invoked.

Hence, we should prevent further requests after being disconnected from owner.
For this bug, we should throw an error instead to prevent invalid access.

Comment 15

[Bevis Tseng[:bevis][:btseng](Exited)]

Attachment: Patch v1: Prevent requests when disconnected from owner window.

Hi reporter,

This patch is to add essential protection before any access to the owner window of the EventTarget.
Would you please help to give it a trial to see if this issue can be fixed?

Thank!

Comment 16

[Julien Wajsberg [:julienw]]

Note that bug 1123853 was filed yesterday, but we can't know if it's the same crash without more information for reporter.

Comment 17

[Bevis Tseng[:bevis][:btseng](Exited)]

(In reply to Julien Wajsberg [:julienw] from comment #16)
> Note that bug 1123853 was filed yesterday, but we can't know if it's the
> same crash without more information for reporter.

No, the call stack looks different from what we saw in this bug in comment 1 and comment 4. :(

Comment 18

[Wei Gao (Spreadtrum)]

(In reply to Bevis Tseng[:bevistseng][:btseng] from comment #15)
> Created attachment 8552281 [details] [diff] [review]
> Patch v1: Prevent requests when disconnected from owner window.
> 
> Hi reporter,
> 
> This patch is to add essential protection before any access to the owner
> window of the EventTarget.
> Would you please help to give it a trial to see if this issue can be fixed?
> 
> Thank!

Dear Bevis

Thanks for your kindly help.
We will have a test tonight. After that, I will tell you the result whatever the outcome.
Thanks again.

Comment 19

[Wei Gao (Spreadtrum)]

Dear Bevis

I am glad to say, this issue doesn't happen again.
But the bad news is another issue occur. 
Could you help to check it together.
Thanks so much.

Operating system: Android
                  0.0.0 Linux 3.10.17 #1 PREEMPT Mon Jan 26 04:17:27 CST 2015 armv7l Spreadtrum/scx15_sp7715eaplus/scx15_sp7715ea:4.4.2/KOT49H/100:userdebug/test-keys
CPU: arm   1 CPU

Crash reason:  SIGSEGV
Crash address: 0x44

Thread 0 (crashed)
 0  libxul.so!mozilla::dom::mobilemessage::MobileMessageCursorCallback::NotifyCursorResult(nsISupports**, unsigned int) [nsTArray.h : 328 + 0x0]
     r4 = 0x00000001    r5 = 0x00000044    r6 = 0xae371200    r7 = 0xbeed3b04
     r8 = 0x010d3972    r9 = 0xae31a610   r10 = 0x00000000    fp = 0xb6b51418
     sp = 0xbeed3ad0    lr = 0xb57946fb    pc = 0xb57921c4
    Found by: given as instruction pointer in context
 1  libxul.so!mozilla::dom::mobilemessage::MobileMessageCursorChild::DoNotifyResult(nsTArray<mozilla::dom::mobilemessage::MobileMessageData> const&) [SmsChild.cpp : 354 + 0xb]
     r4 = 0xbeed3af8    r5 = 0xbeed3b08    r6 = 0x00000001    r7 = 0x00000001
     r8 = 0xbeed3b58    r9 = 0xae31a610   r10 = 0x00000000    fp = 0xb6b51418
     sp = 0xbeed3ae8    pc = 0xb57946fb
    Found by: call frame info
 2  libxul.so!mozilla::dom::mobilemessage::MobileMessageCursorChild::RecvNotifyResult(mozilla::dom::mobilemessage::MobileMessageCursorData const&) [SmsChild.cpp : 298 + 0x3]
     r4 = 0xae31a610    r5 = 0xbeed3b68    r6 = 0x00000000    r7 = 0x00660001
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6b51418
     sp = 0xbeed3b38    pc = 0xb57947eb
    Found by: call frame info
 3  libxul.so!mozilla::dom::mobilemessage::PMobileMessageCursorChild::OnMessageReceived(IPC::Message const&) [PMobileMessageCursorChild.cpp : 179 + 0x9]
     r4 = 0xae31a610    r5 = 0xbeed3b68    r6 = 0x00000000    r7 = 0x00660001
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6b51418
     sp = 0xbeed3b40    pc = 0xb52620ed
    Found by: call frame info
 4  libxul.so!mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) [PContentChild.cpp : 3982 + 0x7]
     r4 = 0xbeed3cd4    r5 = 0xb6b51448    r6 = 0xaf36edf0    r7 = 0xbeed3edc
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6b51418
     sp = 0xbeed3b80    pc = 0xb5219441
    Found by: call frame info
 5  libxul.so!mozilla::ipc::MessageChannel::DispatchAsyncMessage(IPC::Message const&) [MessageChannel.cpp : 1233 + 0x5]
     r4 = 0xbeed3cd4    r5 = 0xb6b51448    r6 = 0xaf36edf0    r7 = 0xbeed3edc
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6865aa0
     sp = 0xbeed3ca0    pc = 0xb51d68ef
    Found by: call frame info
 6  libxul.so!mozilla::ipc::MessageChannel::OnMaybeDequeueOne() [MessageChannel.cpp : 1098 + 0x3]
     r4 = 0x00000001    r5 = 0xbeed3ed0    r6 = 0xaf36edf0    r7 = 0xbeed3edc
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6865aa0
     sp = 0xbeed3cb8    pc = 0xb51d8a73
    Found by: call frame info
 7  libxul.so!RunnableMethod<FdWatcher, void (FdWatcher::*)(), Tuple0>::Run() [tuple.h : 383 + 0x13]
     r4 = 0xae348a48    r5 = 0xbeed3ed0    r6 = 0xaf36edf0    r7 = 0xbeed3edc
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6865aa0
     sp = 0xbeed3cf8    pc = 0xb506e055
    Found by: call frame info
 8  libxul.so!mozilla::ipc::MessageChannel::DequeueTask::Run() [MessageChannel.h : 411 + 0x9]
     r4 = 0xae348a48    r5 = 0xbeed3ed0    r6 = 0xaf36edf0    r7 = 0xbeed3edc
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6865aa0
     sp = 0xbeed3d08    pc = 0xb51d634b
    Found by: call frame info
 9  libxul.so!MessageLoop::RunTask(Task*) [message_loop.cc : 362 + 0x5]
     r4 = 0xae348a48    r5 = 0xbeed3ed0    r6 = 0xaf36edf0    r7 = 0xbeed3edc
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6865aa0
     sp = 0xbeed3d10    pc = 0xb51cd819
    Found by: call frame info
10  libxul.so!MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask const&) [message_loop.cc : 370 + 0x5]
     r4 = 0x00000001    r5 = 0xbeed3d40    r6 = 0xaf36edf0    r7 = 0xbeed3edc
     r8 = 0x00000000    r9 = 0xbeed3dcf   r10 = 0x00000000    fp = 0xb6865aa0
     sp = 0xbeed3d20    pc = 0xb51cdef7
    Found by: call frame info

Comment 20

[Wei Gao (Spreadtrum)]

B2G_OS_Version=2.1.0.0-prerelease
Android_Device=scx15_sp7715ea
Android_Manufacturer=Spreadtrum
ProductName=B2G
Android_Board=scx15_sp7715ea
Android_CPU_ABI=armeabi-v7a
Vendor=Mozilla
InstallTime=1422248852
Notes=GL Layers! EGL? EGL+ GL Context? GL Context+ GL Layers+ 
ReleaseChannel=default
Android_CPU_ABI2=armeabi
Version=34.0
Android_Brand=Spreadtrum
ServerURL=https://crash-reports.mozilla.com/submit?id={3c2e2abc-06d4-11e1-ac3b-374f68613e61}&version=34.0&buildid=20150126041349
Android_Hardware=scx15
useragent_locale=en-US
BuildID=20150126041349
ProductID={3c2e2abc-06d4-11e1-ac3b-374f68613e61}
Android_Version=19(REL)
Android_Model=SP7715A
CrashTime=1422288993
StartupTime=1422261099
ProcessType=
URL=app://sms.gaiamobile.org/manifest.webapp

Comment 21

[Bevis Tseng[:bevis][:btseng](Exited)]

Set ni to myself to follow up.

Comment 22

[Bevis Tseng[:bevis][:btseng](Exited)]

For the crash in comment 19, the crash was happened at [nsTArray.h : 328 + 0x0] nsTArray::Length()[1], which looks quick abnormal from the design of nsTArray because the 'mHdr' will never be null per comments in [2]. 

See also 1126133 for the same problem.

[1] https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/file/c694578ff69e/xpcom/glue/nsTArray.h#l328
[2] https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/file/c694578ff69e/xpcom/glue/nsTArray.h#l447

Comment 23

[Steven Yang [:styang]]

mark 2.1S? for tracking.

Comment 24

[Steven Yang [:styang]]

didn't happen for weeks, minus it. will re-visit once happened again.

Comment 25

[Wei Gao (Spreadtrum)]

I think we can fix this bug as the crash issue in Comment1 doesn't happen again after merging 
"Patch v1: Prevent requests when disconnected from owner window.".

Thanks.