Closed Bug 1090206 Opened 10 years ago Closed 8 years ago

Intermittent testAddonManager | application crashed [@ libc.so + 0x15ec8]

Categories

(Firefox for Android Graveyard :: Add-on Manager, defect)

Product:
Firefox for Android Graveyard
Component:
Add-on Manager
Platform:
x86
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: cbook, Assigned: gbrown)

References

(
URL
)

Details

(Keywords: crash, intermittent-failure) 

Android 2.3 Emulator mozilla-inbound opt test robocop-1

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=3351120&repo=mozilla-inbound

4:34:37 WARNING - PROCESS-CRASH | testAddonManager | application crashed [@ libc.so + 0x15ec8]
04:34:37 INFO - Crash dump filename: /tmp/tmpmchkMo/1a07cc6f-136c-15bf-441e42c1-6c17dc78.dmp
04:34:37 INFO - Operating system: Android
04:34:37 INFO - 0.0.0 Linux 2.6.29-ge3d684d #1 Mon Dec 16 22:26:51 UTC 2013 armv7l generic/sdk/generic:2.3.7/GINGERBREAD/eng.ubuntu.20140123.014351:eng/test-keys
04:34:37 INFO - CPU: arm
04:34:37 INFO - 0 CPUs
04:34:37 INFO -
04:34:37 INFO - Crash reason: SIGSEGV
04:34:37 INFO - Crash address: 0xdeadbaad
04:34:37 INFO -
04:34:37 INFO - Thread 12 (crashed)
04:34:37 INFO - 0 libc.so + 0x15ec8
04:34:37 INFO - r4 = 0x00000001 r5 = 0x00000000 r6 = 0x0033a360 r7 = 0xad9930ad
04:34:37 INFO - r8 = 0x47affb3c r9 = 0x46d61e40 r10 = 0x0022adb8 fp = 0xaca9f368
04:34:37 INFO - sp = 0x47aff988 lr = 0xafd19401 pc = 0xafd15ec8
04:34:37 INFO - Found by: given as instruction pointer in context
04:34:37 INFO - 1 dalvik-heap (deleted) + 0x652196
04:34:37 INFO - sp = 0x47aff9a4 pc = 0x4065b198
04:34:37 INFO - Found by: stack scanning
04:34:37 INFO - 2 libicui18n.so + 0xedac6
04:34:37 INFO - sp = 0x47aff9a8 pc = 0xad9edac8
04:34:37 INFO - Found by: stack scanning
04:34:37 INFO - 3 libc.so + 0x13cad
04:34:37 INFO - sp = 0x47aff9b0 pc = 0xafd13caf
04:34:37 INFO - Found by: stack scanning
04:34:37 INFO - 4 app_process + 0x1c45
04:34:37 INFO - sp = 0x47aff9b8 pc = 0x00009c47
04:34:37 INFO - Found by: stack scanning
04:34:37 INFO - 5 libicuuc.so + 0xc075a
04:34:37 INFO - sp = 0x47aff9bc pc = 0xadec075c
04:34:37 INFO - Found by: stack scanning
04:34:37 INFO - 6 libc.so + 0x14069
04:34:37 INFO - sp = 0x47aff9c8 pc = 0xafd1406b
04:34:37 INFO - Found by: stack scanning
This test is working fine:

04:34:37 INFO - Passed: 26
04:34:37 INFO - Failed: 0
04:34:37 INFO - Todo: 0 

If we want to disable something, disable the pile of fail around the dump crash
Flags: needinfo?(jgriffin)
Flags: needinfo?(gbrown)
(In reply to Mark Finkle (:mfinkle) from comment #6)
> This test is working fine:
> 
> 04:34:37 INFO - Passed: 26
> 04:34:37 INFO - Failed: 0
> 04:34:37 INFO - Todo: 0 
> 
> If we want to disable something, disable the pile of fail around the dump
> crash

This is actually a shutdown crash, so I agree we'd be lucky if disabling this particular test helped.  But I'm not sure what you mean about "disabling the pile of fail around the dump crash".  Generally, the only thing to do here is for developers to figure out what's going wrong. Mark, can you clarify what you're asking for here?

Orthogonally, we should denote these shutdown crashes better, so they don't appear to be related to a specific test.  I'll file a separate bug for that.
Flags: needinfo?(jgriffin) → needinfo?(mark.finkle)
(In reply to TBPL Robot from comment #4)
> submit_timestamp: 2014-10-28T12:18:41
> log:
> https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-
> inbound&job_id=3363415

Some of these crashed in SwapBuffers, like bug 1025968. When we have encountered frequent SwapBuffers crashes in other tests, we have disabled the tests on Android 2.3 -- it seemed like the best we could do. :(

10:46:02  WARNING -  PROCESS-CRASH | testAddonManager | application crashed [@ libc.so + 0x15ec8]
10:46:02     INFO -  Crash dump filename: /tmp/tmpi2ZmgM/621381f0-4159-91a8-7f665af6-3665c909.dmp
10:46:02     INFO -  Operating system: Android
10:46:02     INFO -                    0.0.0 Linux 2.6.29-ge3d684d #1 Mon Dec 16 22:26:51 UTC 2013 armv7l generic/sdk/generic:2.3.7/GINGERBREAD/eng.ubuntu.20140123.014351:eng/test-keys
10:46:02     INFO -  CPU: arm
10:46:02     INFO -       0 CPUs
10:46:02     INFO -  
10:46:02     INFO -  Crash reason:  SIGSEGV
10:46:02     INFO -  Crash address: 0xdeadbaad
10:46:02     INFO -  
10:46:02     INFO -  Thread 41 (crashed)
10:46:02     INFO -   0  libc.so + 0x15ec8
10:46:02     INFO -       r4 = 0x00000001    r5 = 0x00000000    r6 = 0x00416c58    r7 = 0x0000a000
10:46:02     INFO -       r8 = 0x00000000    r9 = 0x00000000   r10 = 0x00000000    fp = 0x00000001
10:46:02     INFO -       sp = 0x56bffa50    lr = 0xafd19401    pc = 0xafd15ec8
10:46:02     INFO -      Found by: given as instruction pointer in context
 ...
10:46:02     INFO -  13  libEGL_emulation.so + 0x71f9
10:46:02     INFO -       sp = 0x56bffb08    pc = 0x804071fb
10:46:02     INFO -      Found by: stack scanning
10:46:02     INFO -  14  libEGL.so + 0x37b7
10:46:02     INFO -       sp = 0x56bffb30    pc = 0xabe037b9
10:46:02     INFO -      Found by: stack scanning
10:46:02     INFO -  15  libxul.so!mozilla::gl::GLContextEGL::SwapBuffers() [GLLibraryEGL.h:91179166c645 : 304 + 0x7]
10:46:02     INFO -       sp = 0x56bffb48    pc = 0x4f27f8cf
10:46:02     INFO -      Found by: stack scanning
10:46:02     INFO -  16  libxul.so!mozilla::layers::CompositorOGL::EndFrame() [CompositorOGL.cpp:91179166c645 : 1207 + 0x5]
10:46:02     INFO -       r4 = 0x571e0160    r5 = 0x00000000    r6 = 0x571e0240    sp = 0x56bffb60
10:46:02     INFO -       pc = 0x4f2cddc5
10:46:02     INFO -      Found by: call frame info
10:46:02     INFO -  17  libxul.so!mozilla::layers::LayerManagerComposite::Render() [LayerManagerComposite.cpp:91179166c645 : 755 + 0x5]
10:46:02     INFO -       r4 = 0x56bffbd4    r5 = 0x571e0258    r6 = 0x571e0240    sp = 0x56bffb98
10:46:02     INFO -       pc = 0x4f2b9da5
10:46:02     INFO -      Found by: call frame info
Depends on: 1025968
Jeff - Just some frustration around shutdown crashes, usually related to something completely different than the test itself. But then the test is disabled for being flakey.

Based on the amount of intermittent failures we are seeing on the emulators, I am worried that emulators are not reliable enough to run our test suites. The number of people who can work on these kinds of failures is very small.

If something in testAddonManager was failing due to an error in the test or an error in the Add-on manager code, I could throw a larger number of people at the problem.

Mood: Frustrated
Flags: needinfo?(mark.finkle)
I'm sympathetic with your frustrations.

Unfortunately, whether the test is at fault or the platform, the sheriffs have little choice but to hide or disable if their failure rate is high.

Is there anyone available to really dig into this problem and figure out what's going on?  Is there a resource we could schedule later this quarter, if not now?
(In reply to Mark Finkle (:mfinkle) from comment #9)
> If something in testAddonManager was failing due to an error in the test or
> an error in the Add-on manager code, I could throw a larger number of people
> at the problem.

Could you throw a few people at bug 1025968? It is the root cause of many Android 2.3 robocop failures and has been my greatest concern since setting up the 2.3 environment. :snorp has looked at it, but more hands and eyes couldn't hurt.

> Based on the amount of intermittent failures we are seeing on the emulators,
> I am worried that emulators are not reliable enough to run our test suites.

I have concerns about the emulators, but in my experience nearly all xpcshell, reftests, and (with the exception of media) mochitests are as reliable on 2.3 as they were on the Tegras. Robocop is not -- we have a ridiculous number of robocop tests skipped on 2.3 and more need to be skipped. Even so, the weekly test failure rates (number of failed vs successful test jobs on m-c) are better on 2.3 than on 4.0. That probably does nothing to improve your mood and there are different ways of looking at reliability, but it's something to keep in mind.

I'd really like to see bug 1025968 resolved; I think that would go a long way to making robocop useful on 2.3.
Flags: needinfo?(gbrown)
Android 2.3 is no longer supported in Firefox 48+.

Test manifests were updated in bug 1251013.
Assignee: nobody → gbrown
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
Whiteboard: [test disabled][leave open]
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.