Closed Bug 613954 Opened 14 years ago Closed 13 years ago

OOM crash [@ libc.so@0x11c80 ][@ libc.so@0x11da0 ][@ libc.so@0x11cf0 ][@ libc.so@0x11ca0 ][@ libc.so@0x11f74 ][@ libc.so@0x11dc0 ][@ libc.so@0x11cd0 ][@ libc.so@0x11f0c ][@ libc.so@0x11e30 ][@ libc.so@0x11f84 ][@ libc.so@0x11cb4 ][@ libc.so@0x123b0 ]

Categories

(Core :: IPC, defect, P1)

ARM
Android
defect

Tracking

()

RESOLVED INCOMPLETE
Tracking Status
fennec - ---

People

(Reporter: scoobidiver, Unassigned)

References

Details

(Keywords: crash)

Crash Data

Attachments

(1 file, 1 obsolete file)

It is #14 top crasher in Fennec 4.0b3pre for the last week.

Signature	libc.so@0x11cf0
UUID	bf885766-31a8-4c01-8f2e-afe212101115
Time 	2010-11-15 04:26:57.800476
Uptime	51591
Install Age	160174 seconds (1.9 days) since version was first installed.
Product	Fennec
Version	4.0b3pre
Build ID	20101113041939
Branch	2.0
OS	Linux
OS Version	0.0.0 Linux 2.6.32.15-g6a358a9 #1 PREEMPT Fri Aug 6 22:36:38 CST 2010 armv7l
CPU	arm
CPU Info	
Crash Reason	SIGSEGV
Crash Address	0xdeadbaad
User Comments	
App Notes 	HTC HTC Desire
htc_wwe/htc_bravo/bravo/bravo:2.2/FRF91/226611:user/release-keys
Processor Notes 	
EMCheckCompatibility	False
Crashing Thread
Frame 	Module 	Signature [Expand] 	Source
0 	libc.so 	libc.so@0x11cf0 	
1 	libc.so 	libc.so@0x18e5a 	
2 	libmozalloc.so 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:75
3 	libxul.so 	StuffFixedBuffer 	xpcom/base/nsDebugImpl.cpp:240
4 	framework.odex 	framework.odex@0x14af6c 	
5 	org.mozilla.fennec-1.apk 	org.mozilla.fennec-1.apk@0x540fff 	
6 	libicudata.so 	libicudata.so@0x7 	
7 	libxul.so 	GeckoStart 	toolkit/xre/nsAndroidStartup.cpp:76
8 	libnspr4.so 	PR_Unlock 	nsprpub/pr/src/pthreads/ptsynch.c:245
9 		@0x1 	
10 	core.odex 	core.odex@0x11e14b 	
11 	core.odex 	core.odex@0x11e161 	
12 	libskia.so 	libskia.so@0x2e9db 	
13 	libdvm.so 	libdvm.so@0x1bd93 	
14 	libdvm.so 	libdvm.so@0x22b47 	
15 	libdvm.so 	libdvm.so@0x21a63 	
16 	libxul.so 	nsDOMEvent::QueryInterface 	content/events/src/nsDOMEvent.cpp:186
17 	libxul.so 	nsDOMUIEvent::QueryInterface 	content/events/src/nsDOMUIEvent.cpp:126
18 		@0x4cc1e27f 	
19 	libxul.so 	NS_CycleCollectorSuspect2_P 	xpcom/base/nsCycleCollector.cpp:3187

More reports at:
http://crash-stats.mozilla.com/report/list?range_value=4&range_unit=weeks&signature=libc.so%400x11cf0&version=Fennec%3A4.0b3pre
blocking2.0: --- → ?
blocking2.0: ? → ---
tracking-fennec: --- → ?
Summary: crash [@ libc.so@0x11cf0 ] → crash [@ libc.so@0x11cf0 ][@ libc.so@0x11c80 ]
Component: XPCOM → General
QA Contact: xpcom → general
sounds like a OOM.
tracking-fennec: ? → 2.0-
Keywords: topcrash
The majority of these crashes in the past few days (~50/day) for 4.0b3 have shared a very common stack:

0 	libc.so 	libc.so@0x11c80 	
1 	libc.so 	libc.so@0x1039b 	
2 	libc.so 	libc.so@0xbdf6 	
3 	libc.so 	libc.so@0xcd54 	
4 	libGLESv2_POWERVR_SGX530_121.so 	libGLESv2_POWERVR_SGX530_121.so@0x130f7 	
5 	libGLESv2_POWERVR_SGX530_121.so 	libGLESv2_POWERVR_SGX530_121.so@0xcecb 	
6 	libxul.so 	mozilla::gl::GLContext::DeleteOffscreenFBO 	gfx/thebes/GLContext.h:1831
7 	libxul.so 	mozilla::gl::GLContext::MarkDestroyed 	gfx/thebes/GLContext.h:1819
8 	libxul.so 	mozilla::gl::GLContextEGL::~GLContextEGL 	gfx/thebes/GLContextProviderEGL.cpp:577
9 	libxul.so 	mozilla::gl::GLContextEGL::~GLContextEGL 	mozalloc.h:253
10 	libxul.so 	mozilla::gl::GLContext::Release 	GLContext.h:361
11 	libxul.so 	nsRefPtr<mozilla::gl::GLContext>::operator= 	nsAutoPtr.h:1027
12 	libxul.so 	mozilla::WebGLContext::DestroyResourcesAndContext 	content/canvas/src/WebGLContext.cpp:233
13 	libxul.so 	mozilla::WebGLContext::~WebGLContext 	nsAutoPtr.h:537
14 	libxul.so 	mozilla::WebGLContext::~WebGLContext 	mozalloc.h:253
15 	libxul.so 	mozilla::WebGLContext::Release 	content/canvas/src/WebGLContext.cpp:703
16 	libxul.so 	nsCOMPtr_base::~nsCOMPtr_base 	nsCOMPtr.cpp:82
17 	libxul.so 	nsHTMLCanvasElement::~nsHTMLCanvasElement 	nsTSubstring.h:113
18 	libxul.so 	nsHTMLCanvasElement::~nsHTMLCanvasElement 	mozalloc.h:253
19 	libxul.so 	nsNodeUtils::LastRelease 	content/base/src/nsNodeUtils.cpp:328
20 	libxul.so 	nsGenericElement::Release 	content/base/src/nsGenericElement.cpp:4486
21 	libxul.so 	nsHTMLCanvasElement::Release 	content/html/content/src/nsHTMLCanvasElement.cpp:88
22 	libxul.so 	nsXPCOMCycleCollectionParticipant::Unroot 	nsCycleCollectionParticipant.cpp:76
23 	libxul.so 	nsCycleCollector::CollectWhite 	xpcom/base/nsCycleCollector.cpp:1931
24 	libxul.so 	nsCycleCollector::FinishCollection 	xpcom/base/nsCycleCollector.cpp:2731
25 	libxul.so 	nsCycleCollectorRunner::Collect 	xpcom/base/nsCycleCollector.cpp:3365
26 	libxul.so 	nsCycleCollector_collect 	xpcom/base/nsCycleCollector.cpp:3473
27 	libxul.so 	nsJSContext::CC 	dom/base/nsJSEnvironment.cpp:3651
28 	libxul.so 	nsJSContext::IntervalCC 	dom/base/nsJSEnvironment.cpp:3755
29 	libxul.so 	nsJSContext::MaybeCC 	dom/base/nsJSEnvironment.cpp:3723
30 	libxul.so 	nsJSContext::MaybeCCIfUserInactive 	dom/base/nsJSEnvironment.cpp:3746
31 	libxul.so 	nsXPConnect::AfterProcessNextEvent 	js/src/xpconnect/src/nsXPConnect.cpp:2311
32 	libxul.so 	nsThread::ProcessNextEvent 	nsCOMPtr.h:492
33 	libxul.so 	NS_ProcessNextEvent_P 	nsThreadUtils.cpp:250
34 	libxul.so 	mozilla::ipc::MessagePump::Run 	ipc/glue/MessagePump.cpp:134
35 	libxul.so 	mozilla::ipc::MessagePumpForChildProcess::Run 	ipc/glue/MessagePump.cpp:230
36 	libxul.so 	MessageLoop::RunInternal 	ipc/chromium/src/base/message_loop.cc:220
37 	libxul.so 	MessageLoop::Run 	ipc/chromium/src/base/message_loop.cc:512
38 	libxul.so 	nsBaseAppShell::Run 	widget/src/xpwidgets/nsBaseAppShell.cpp:198
39 	libxul.so 	XRE_RunAppShell 	toolkit/xre/nsEmbedFunctions.cpp:631
40 	libxul.so 	mozilla::ipc::MessagePumpForChildProcess::Run 	ipc/glue/MessagePump.cpp:222
41 	libxul.so 	MessageLoop::RunInternal 	ipc/chromium/src/base/message_loop.cc:220
42 	libxul.so 	MessageLoop::Run 	ipc/chromium/src/base/message_loop.cc:512
43 	libxul.so 	XRE_InitChildProcess 	toolkit/xre/nsEmbedFunctions.cpp:510
44 	libmozutils.so 	ChildProcessInit 	other-licenses/android/APKOpen.cpp:691
45 	plugin-container 	main 	ipc/app/MozillaRuntimeMainAndroid.cpp:69
46 	libc.so 	libc.so@0xd3c2
Stack traces in comment 2 are related to WebGL and are not the main crash reason for this bug. I filed bug 623174 for stack traces in comment 2.
For this bug, the first 4 frames are mainly the same as in comment 0.

With combined signatures, it is #4 top crasher in 4.0b3 for the last week.

More reports at:
http://crash-stats.mozilla.com/query/query?product=Fennec&range_value=4&range_unit=weeks&query_search=signature&query_type=startswith&query=libc.so@0x11&build_id=&process_type=any&hang_type=any&do_query=1
Summary: crash [@ libc.so@0x11cf0 ][@ libc.so@0x11c80 ] → Fennec OOM crash [@ libc.so@0x11c80 ][@ libc.so@0x11da0 ][@ libc.so@0x11cf0 ][@ libc.so@0x11ca0 ][@ libc.so@0x11f74 ][@ libc.so@0x11dc0 ][@ libc.so@0x11cd0 ]
Summary: Fennec OOM crash [@ libc.so@0x11c80 ][@ libc.so@0x11da0 ][@ libc.so@0x11cf0 ][@ libc.so@0x11ca0 ][@ libc.so@0x11f74 ][@ libc.so@0x11dc0 ][@ libc.so@0x11cd0 ] → Fennec OOM crash [@ libc.so@0x11c80 ][@ libc.so@0x11da0 ][@ libc.so@0x11cf0 ][@ libc.so@0x11ca0 ][@ libc.so@0x11f74 ][@ libc.so@0x11dc0 ][@ libc.so@0x11cd0 ][@ libc.so@0x11f0c ][@ libc.so@0x11e30 ][@ libc.so@0x11f84 ][@ libc.so@0x11cb4 ]
Summary: Fennec OOM crash [@ libc.so@0x11c80 ][@ libc.so@0x11da0 ][@ libc.so@0x11cf0 ][@ libc.so@0x11ca0 ][@ libc.so@0x11f74 ][@ libc.so@0x11dc0 ][@ libc.so@0x11cd0 ][@ libc.so@0x11f0c ][@ libc.so@0x11e30 ][@ libc.so@0x11f84 ][@ libc.so@0x11cb4 ] → OOM crash [@ libc.so@0x11c80 ][@ libc.so@0x11da0 ][@ libc.so@0x11cf0 ][@ libc.so@0x11ca0 ][@ libc.so@0x11f74 ][@ libc.so@0x11dc0 ][@ libc.so@0x11cd0 ][@ libc.so@0x11f0c ][@ libc.so@0x11e30 ][@ libc.so@0x11f84 ][@ libc.so@0x11cb4 ][@ libc.so@0x123b0 ]
Here is another kind of stack traces:
0 	libc.so 	libc.so@0x11c80 	
1 	libc.so 	libc.so@0x18dea 	
2 	libmozalloc.so 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:75
3 	libxul.so 	StuffFixedBuffer 	xpcom/base/nsDebugImpl.cpp:240
4 	org.mozilla.fennec-1.apk 	org.mozilla.fennec-1.apk@0x9db035 	
5 	libc.so 	libc.so@0x1027f 	
6 	libc.so 	libc.so@0xbe0c 	
7 	libbinder.so 	libbinder.so@0x1aab2 	
8 	libc.so 	libc.so@0xcd54 	
9 	libxul.so 	nsRegion::SetToElements 	gfx/src/nsRegion.cpp:294
10 		@0x4a89a89b 	
11 	libxul.so 	nsRegion::SubRect 	gfx/src/nsRegion.cpp:1248
12 	libxul.so 	Pickle::AlignInt 	ipc/chromium/src/base/pickle.h:272
13 	libxul.so 	Pickle::UpdateIter 	ipc/chromium/src/base/pickle.h:278
14 	libxul.so 	Pickle::ReadInt 	ipc/chromium/src/base/pickle.cc:145
15 	libxul.so 	IPC::ReadParam<nsIntRect> 	ipc/chromium/src/chrome/common/ipc_message_utils.h:187
16 		@0x3 	
17 	libxul.so 	Pickle::AlignInt 	ipc/chromium/src/base/pickle.h:272
18 	libxul.so 	Pickle::UpdateIter 	ipc/chromium/src/base/pickle.h:278
19 	libxul.so 	Pickle::ReadInt 	ipc/chromium/src/base/pickle.cc:145

I suspect an IPC issue. But may be these OOM crashes can be caused for different reasons.
Component: General → IPC
QA Contact: general → ipc
Blocks: 628715
Mozilla/5.0 (Android; Linux armv71; rv2.0b12pre) Gecko/20110216 Firefox/4.0b12pre Fennec/4.0b5pre
Device: Droid 2 
OS: Android 2.2

Found a repro step:

1. go to : http://www.google.com/m?q=bloomberg
Expected: no content crash
Actual: content crash

Note:
1: https://crash-stats.mozilla.com/report/index/bp-155c724b-106a-4877-a89c-ce8f12110216
Fennec 4.0b5pre Crash Report [@ libc.so@0x11cd0 ]
Note : I was trying to do a google search using the awesome page
I can hit this each time i visit planet.mozilla.org on Nexus S.

This is on android nightly build: Mozilla/5.0 (Android; Linux armv71; rv:2.0b12pre) Gecko/20110216 Firefox/4.0b12pre Fennec/4.0b5pre
I got this while loading planet.mozilla.org on my nexus one

http://crash-stats.mozilla.com/report/index/c1ad684b-6645-410d-99fa-c25a02110216

0 	libc.so 	libc.so@0x11c80 	
1 	libc.so 	libc.so@0x18dea 	
2 	libmozalloc.so 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:75
3 	org.mozilla.fennec-2.apk 	org.mozilla.fennec-2.apk@0x47a120 	
4 	libxul.so 	nsNodeIterator::cycleCollection::Traverse 	content/base/src/nsNodeIterator.cpp:200
5 		@0x6b636161 	
6 	libxul.so 	nsGlobalWindow::RunTimeout 	dom/base/nsGlobalWindow.cpp:9073
7 		@0x41c2b0bf
Some of these look like OOM aborts, but the one from comment 9 looks very much like an NS_RUNTIMEABORT() exception.  Around 2/2 we got code to annotate RUNTIMEABORT()s in bug 628885, but I don't see one of those annotations here.  They're supposed to be appear in the "App Notes" field in crashs reports but I don't see that field at all.  I wonder if that's a crashreporter/breakpad bug.  There are tests that would catch this, but we don't run them on android.

The SIGSEGV@0xdeadbaad below mozalloc_abort() is hilarious, turns out the bionic code that follows is the reason

void
abort(void)
#endif
{
	struct atexit *p = __atexit;
	static int cleanup_called = 0;
	sigset_t mask;

  
	sigfillset(&mask);
	/*
	 * don't block SIGABRT to give any handler a chance; we ignore
	 * any errors -- X311J doesn't allow abort to return anyway.
	 */
	sigdelset(&mask, SIGABRT);
    /* temporary, so deliberate seg fault can be caught by debuggerd */
	sigdelset(&mask, SIGSEGV);
    /* -- */
	(void)sigprocmask(SIG_SETMASK, &mask, (sigset_t *)NULL);

	/*
	 * POSIX requires we flush stdio buffers on abort
	 */
	if (cleanup_called == 0) {
		while (p != NULL && p->next != NULL)
			p = p->next;
		/* the check for fn_dso == NULL is mostly paranoia */
		if (p != NULL && p->fns[0].fn_dso == NULL &&
		    p->fns[0].fn_ptr.std_func != NULL) {
			cleanup_called = 1;
			(*p->fns[0].fn_ptr.std_func)();
		}
	}

    /* temporary, for bug hunting */
    /* seg fault seems to produce better debuggerd results than SIGABRT */
    *((char*)0xdeadbaad) = 39;
    /* -- */

	(void)kill(getpid(), SIGABRT);

Keep it classy, android.
(Couldn't they have at least used 0xa80127?)
I reproduced this with a debug build and got the following in the log:

I/Gecko   ( 4823): nsWindow[0x4db20f20]::DrawTo child[0]
I/Gecko   ( 4823): nsWindow[0x4db21ef0]::DrawTo no covering child, drawing this
I/Gecko   ( 4823): nsWindow[0x4db21ef0]::DrawTo done
I/Gecko   ( 4823): nsWindow[0x4db20f20]::DrawTo done
I/Gecko   ( 4841): ###!!! ABORT: creating ThebesLayer 'back buffer' failed!: file ../../../gfx/layers/basic/BasicLayers.cpp, line 1817
I/Gecko   ( 4823): nsWindow::Invalidate 0x4db21ef0 [23 24 33 32]
I/Gecko   ( 4823): >> OnDraw
I/Gecko   ( 4823): nsWindow[0x4db20f20]::DrawTo child[0]
I/Gecko   ( 4823): nsWindow[0x4db21ef0]::DrawTo no covering child, drawing this
I/DEBUG   ( 1054): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   ( 1054): Build fingerprint: 'google/passion/passion/mahimahi:2.2.2/FRG83G/91102:user/release-keys'
I/DEBUG   ( 1054): pid: 4841, tid: 4841  >>> /data/data/org.mozilla.fennec/plugin-container <<<
I/DEBUG   ( 1054): signal 11 (SIGSEGV), fault addr deadbaad
I/DEBUG   ( 1054):  r0 00000000  r1 afd14629  r2 00000027  r3 00000070
I/DEBUG   ( 1054):  r4 afd42328  r5 00000000  r6 00000000  r7 beaa98f0
I/DEBUG   ( 1054):  r8 00000000  r9 00000000  10 00000000  fp 00000000
I/DEBUG   ( 1054):  ip 00001728  sp beaa98a8  lr deadbaad  pc afd11c80  cpsr 60000030
That's a good old-fashioned OOM abort

I/Gecko   ( 4841): ###!!! ABORT: creating ThebesLayer 'back buffer' failed!:
file ../../../gfx/layers/basic/BasicLayers.cpp, line 1817
(In reply to comment #13)
> That's a good old-fashioned OOM abort
> 
> I/Gecko   ( 4841): ###!!! ABORT: creating ThebesLayer 'back buffer' failed!:
> file ../../../gfx/layers/basic/BasicLayers.cpp, line 1817

yup, I added another logging statement just before this:

E/GeckoLayers( 4968): failed to create a 480 x 33880 double buffer

so, that's 2 480x33880 32bit buffers, right? so roughly 124MB. I can see how that would fail. Did this change recently?
are either of you also seeing many dozen of reallocations due to small displayport changes?
(In reply to comment #14)
> E/GeckoLayers( 4968): failed to create a 480 x 33880 double buffer
> 
> so, that's 2 480x33880 32bit buffers, right?

o_O.  That smells like some displayport calculation gone wrong.  That should be a separate bug filed against Fennec:Panning and Zooming.
yup.  filed.  thanks.
Comment on attachment 513178 [details] [diff] [review]
OOM crash related to displayport

We had a very bad ratio calculation that affects very long or very wide pages.
Attachment #513178 - Flags: review?(mbrubeck)
Attachment #513178 - Flags: review?(mbrubeck) → review+
(In reply to comment #10)
> They're supposed to be appear in the "App Notes" field in crashs reports but I
> don't see that field at all.  I wonder if that's a crashreporter/breakpad bug. 
> There are tests that would catch this, but we don't run them on android.

We're not forwarding these from content processes.  Pretty easy fix, will spin off another bug.
Comment on attachment 513178 [details] [diff] [review]
OOM crash related to displayport

Moved At Chris Jones request to bug 634788.
Attachment #513178 - Attachment is obsolete: true
Attached file logcat of crash
crash also occurs when:
1. going to http://ie.microsoft.com/testdrive/HTML5/Geolocation/Default.html
2. clicking on the Locate Me! button
3. click on Share for geolocation notification

Expected: no crash
Actual: crash
Comment 22 based on:
Mozilla/5.0 (Android; Linux armv71; rv2.0b13pre) Gecko/20110310 Firefox/4.0b13pre Fennec/4.0b6pre
Device: Droid 2 
OS: Android 2.2
Attachment #518527 - Attachment mime type: application/octet-stream → text/plain
Crash Signature: [@ libc.so@0x11c80 ] [@ libc.so@0x11da0 ] [@ libc.so@0x11cf0 ] [@ libc.so@0x11ca0 ] [@ libc.so@0x11f74 ] [@ libc.so@0x11dc0 ] [@ libc.so@0x11cd0 ] [@ libc.so@0x11f0c ] [@ libc.so@0x11e30 ] [@ libc.so@0x11f84 ] [@ libc.so@0x11cb4 ] [@ libc.so@0x123b0 ]
Naoki, what should happen to this bug. I know that we have more symbols and these have likely been logged as other bugs correct? Is it valuable to keep this bug open? Is it still a top crash?
Crash Signature: [@ libc.so@0x11c80 ] [@ libc.so@0x11da0 ] [@ libc.so@0x11cf0 ] [@ libc.so@0x11ca0 ] [@ libc.so@0x11f74 ] [@ libc.so@0x11dc0 ] [@ libc.so@0x11cd0 ] [@ libc.so@0x11f0c ] [@ libc.so@0x11e30 ] [@ libc.so@0x11f84 ] [@ libc.so@0x11cb4 ] [@ libc.so@0x123b0 ] → libc.so@0x123b0 ] [@ libc.so@0x11c80 ] [@ libc.so@0x11da0 ] [@ libc.so@0x11cf0 ] [@ libc.so@0x11ca0 ] [@ libc.so@0x11f74 ] [@ libc.so@0x11dc0 ] [@ libc.so@0x11cd0 ] [@ libc.so@0x11f0c ] [@ libc.so@0x11e30 ] [@ libc.so@0x11f84 ] [@ libc.so@0x11cb…
As libc.so has been added to the Socorro skiplist, those crashes are now tracked by bugs with mozalloc_abort signatures.
I close it as incomplete.
Status: NEW → RESOLVED
Closed: 13 years ago
Keywords: topcrash
Resolution: --- → INCOMPLETE
(In reply to Scoobidiver from comment #26)
> As libc.so has been added to the Socorro skiplist, those crashes are now
> tracked by bugs with mozalloc_abort signatures.
> I close it as incomplete.

It's not the skiplist but the Android Symbol Sender that caused that change, but you are basically right. We intentionally have left this to Naoki though, as he has an overview of all the mobile crashes. Please be careful in such cases, as we are handling Fennec Android crashes with different scrutiny than Firefox desktop crashes.
The skiplist is composed of irrelevantSignatureRegEx and prefixSignatureRegEx. libc.so belongs to irrelevantSignatureRegEx, so that it no longer appears in the crash signature.
What you say is true, but I think the libc frames have actually been replaced by resolved symbols in libc for the most part. Still, I agree that this bug should probably be closed, but I still want to hear what Naoki says.
I agree with Scoobidiver.  

The only way that we can resolve the symbols in libc at this time is if people install the android symbol sender that Ted had worked on or if a developer ends up crashing with the same crash.  

Note, some of these libc crashes will correlate to already existing symbols and bugs that have been reported.  Also there could potentially be multiple different crashes hiding behind a single libc crash signature as well.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: