720353 - Crash with abort message "xpcom_runtime_abort(###!!! ABORT: unknown union type: file /builds/slave/rel-m-beta-lnx-andrd-bld/build/obj-firefox/ipc/ipdl/PLayersChild.cpp, line 550)"

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Scoobidiver (away)]

It's a new crash signature that first appeared in Fennec 10.0b5.
It's currently #1 top crasher in 10.0b5 with 70% (90/129) of all crashes.

Many stacks are buggy but it might be a Graphics or Layout bug.
In the Beta regression range:
http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=69368d1fa5bf&tochange=11d741e4641c
it might be caused by bug 694964 or bug 715916.

The number of crashes caused by this bug (0.52 crash/ADU) is higher than the one fixed by bug 694964 (0.24 crash/ADU):
https://crash-stats.mozilla.com/daily?form_selection=by_version&p=Fennec&v[]=10.0b4&throttle[]=100.00&v[]=10.0b5&throttle[]=100.00&v[]=&throttle[]=100.00&v[]=&throttle[]=100.00&hang_type=any&os[]=Windows&os[]=Mac&os[]=Linux&date_start=2012-01-08&date_end=2012-01-24&submit=Generate

Frame 	Module 	Signature 	Source
0 	libmozalloc.so 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:66
1 	libc.so 	__swrite 	
2 	libxul.so 	libxul.so@0xe7cd58 	
3 	libxul.so 	nsWSRunObject::DeleteChars 	editor/libeditor/html/nsWSRunObject.cpp:1607
4 		@0x1 	
5 	libxul.so 	_cairo_path_fixed_add 	gfx/cairo/cairo/src/cairo-path-fixed.c:775
6 	libxul.so 	_cairo_path_fixed_move_to 	gfx/cairo/cairo/src/cairo-path-fixed.c:414
7 	libxul.so 	_cairo_path_fixed_close_path 	gfx/cairo/cairo/src/cairo-path-fixed.c:659
8 	libxul.so 	_moz_cairo_close_path 	gfx/cairo/cairo/src/cairo.c:2164
9 	libxul.so 	_moz_cairo_rectangle 	gfx/cairo/cairo/src/cairo.c:2109
10 	libxul.so 	gfxContext::Rectangle 	gfx/thebes/gfxContext.cpp:259
11 	libxul.so 	gfxSurfaceDrawable::Draw 	gfx/thebes/gfxDrawable.cpp:181
12 		@0x6 	
13 	libmozutils.so 	malloc_mutex_unlock 	memory/jemalloc/jemalloc.c:1539
14 	libmozutils.so 	arena_dalloc 	memory/jemalloc/jemalloc.c:4510
15 		@0x2 	
16 	libxul.so 	Pickle::WriteBytes 	ipc/chromium/src/base/pickle.cc:423
17 		@0xbebbddea 	
18 	libxul.so 	mozilla::layers::PLayersChild::Write 	obj-firefox/ipc/ipdl/PLayersChild.cpp:550
19 	libxul.so 	mozilla::layers::PLayersChild::Write 	obj-firefox/ipc/ipdl/PLayersChild.cpp:1709
20 	libxul.so 	mozilla::layers::PLayersChild::Write 	obj-firefox/ipc/ipdl/PLayersChild.cpp:635
21 	libxul.so 	mozilla::layers::PLayersChild::Write 	obj-firefox/ipc/ipdl/PLayersChild.cpp:1048
22 	libxul.so 	mozilla::layers::PLayersChild::Write 	obj-firefox/ipc/ipdl/PLayersChild.cpp:962
23 	libxul.so 	mozilla::layers::PLayersChild::SendUpdate 	obj-firefox/ipc/ipdl/PLayersChild.cpp:156
24 	libxul.so 	mozilla::layers::ShadowLayerForwarder::EndTransaction 	gfx/layers/ipc/ShadowLayers.cpp:324
25 	libxul.so 	mozilla::layers::BasicShadowLayerManager::ForwardTransaction 	gfx/layers/basic/BasicLayers.cpp:3353
26 	libxul.so 	mozilla::layers::BasicShadowLayerManager::EndTransaction 	gfx/layers/basic/BasicLayers.cpp:3328
27 	libxul.so 	nsDisplayList::PaintForFrame 	layout/base/nsDisplayList.cpp:633
28 	libxul.so 	nsDisplayList::PaintRoot 	layout/base/nsDisplayList.cpp:538
29 	libxul.so 	nsLayoutUtils::PaintFrame 	layout/base/nsLayoutUtils.cpp:1700
30 	libxul.so 	PresShell::Paint 	layout/base/nsPresShell.cpp:5475
31 	libxul.so 	nsViewManager::RenderViews 	view/src/nsViewManager.cpp:415
32 	libxul.so 	nsViewManager::Refresh 	view/src/nsViewManager.cpp:390
33 	libxul.so 	nsViewManager::DispatchEvent 	view/src/nsViewManager.cpp:888
34 	libxul.so 	HandleEvent 	view/src/nsView.cpp:159
35 	libxul.so 	mozilla::widget::PuppetWidget::DispatchEvent 	widget/src/xpwidgets/PuppetWidget.cpp:320
36 	libxul.so 	mozilla::widget::PuppetWidget::DispatchPaintEvent 	widget/src/xpwidgets/PuppetWidget.cpp:561
37 	libxul.so 	mozilla::widget::PuppetWidget::PaintTask::Run 	widget/src/xpwidgets/PuppetWidget.cpp:601
38 	libxul.so 	nsThread::ProcessNextEvent 	xpcom/threads/nsThread.cpp:631
39 	libxul.so 	NS_ProcessNextEvent_P 	obj-firefox/xpcom/build/nsThreadUtils.cpp:245
...

Comment 1

[Ali Juma [:ajuma]]

(In reply to Scoobidiver from comment #0)
> It's a new crash signature that first appeared in Fennec 10.0b5.
> It's currently #1 top crasher in 10.0b5 with 70% (90/129) of all crashes.
> 
> Many stacks are buggy but it might be a Graphics or Layout bug.
> In the Beta regression range:
> http://hg.mozilla.org/releases/mozilla-beta/
> pushloghtml?fromchange=69368d1fa5bf&tochange=11d741e4641c
> it might be caused by bug 694964 or bug 715916.

In this regression range, Bug 694964 is the most likely cause.


> The number of crashes caused by this bug (0.52 crash/ADU) is higher than the
> one fixed by bug 694964 (0.24 crash/ADU):

In light of this, we should back out Bug 694964 from beta ASAP.

Comment 2

[Ali Juma [:ajuma]]

Bug 694964 has been backed out from Beta, which should fix this crash.

It has not been backed out from Aurora or m-c, so we should keep an eye out for this crash happening there.

Comment 3

[Scoobidiver (away)]

There have been no crashes in Fennec 10.0b6.

Comment 4

[Alex Keybl [:akeybl]]

(In reply to Ali Juma [:ajuma] from comment #2)
> Bug 694964 has been backed out from Beta, which should fix this crash.
> 
> It has not been backed out from Aurora or m-c, so we should keep an eye out
> for this crash happening there.

We've backed out bug 694964 from Beta 11 as well.

Comment 5

[Scoobidiver (away)]

Let's reopen it as it's not fixed in XUL Fx 12 and above.

Comment 6

[Oleg Romashin (:romaxa)]

Not sure, but it looks like BasicShadowableThebesLayer::PaintBuffer called twice, and first time we set mBackBuffer = SurfaceDescriptor();
and second time we just abort here:
http://mxr.mozilla.org/mozilla-central/source/gfx/layers/basic/BasicLayers.cpp#2374

Comment 7

[Oleg Romashin (:romaxa)]

btw, do we have syslog output for this crash or something like that, if there are "should have a back buffer by now" message, then my assumption is correct and we have to do something with that abort...

Comment 8

[Scoobidiver (away)]

The back buffer log is in 11.0 but there are only 9 ADU in FennecAndroid 11.0b1 and 0 ADU in XUL Fennec 11.0b1 so far and nobody hit that crash.

Comment 9

[Oleg Romashin (:romaxa)]

Ok, I took current beta 11, applied patches from 694964, and cannot reproduce this crash.
And IIUC crash happening on IPDL attempt to process thebesLayer Swap transaction, which means that we already passed http://mxr.mozilla.org/mozilla-central/source/gfx/layers/basic/BasicLayers.cpp#2374 - surface check point, and that is possible only in next chain

1) BasicShadowableThebesLayer::PaintBuffer - > thebesPaint pushed into transactions array and related buffer assigned to mROFrontBuffer
2) BasicShadowableThebesLayer::SetBackBufferAndAttrs call came from previous transaction. and drop last reference to buffer which is in pending transactions queue by mROFrontBuffer = aReadOnlyFrontBuffer.
3) Result SendUpdate has transaction with ThebesSwap and invalid buffer

Comment 10

[Oleg Romashin (:romaxa)]

I think we should modify fix for bug 694964, and Hold reference to buffer which is pending transaction queue in different variable and do not use mROFrontBuffer

Comment 11

[Ali Juma [:ajuma]]

I'm sometimes running into this now on Native Fennec. We should back out bug 694964 from m-c.

Comment 12

[Ali Juma [:ajuma]]

Attachment: Backout of Bug 694964.

Comment 13

[Ali Juma [:ajuma]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/af19e5ada310

Comment 14

[Mounir Lamouri (:mounir)]

https://hg.mozilla.org/mozilla-central/rev/af19e5ada310

Comment 15

[Alex Keybl [:akeybl]]

This is presumably still affecting FF10 XUL Fennec builds (10.0.3 was pushed to the market on 3/13). Can we uplift the fix to the ESR, from which XUL Fennec is building for releases?

If we think this affects Fennec Native as well, we should also uplift to Aurora 13.

Comment 16

[Ali Juma [:ajuma]]

(In reply to Alex Keybl [:akeybl] from comment #15)
> This is presumably still affecting FF10 XUL Fennec builds (10.0.3 was pushed
> to the market on 3/13). Can we uplift the fix to the ESR, from which XUL
> Fennec is building for releases?

Bug 694964 was backed out of FF10 on Jan. 23 (see Comment 2 above), so this crash shouldn't be happening there.

> If we think this affects Fennec Native as well, we should also uplift to
> Aurora 13.

Yes, this affects Fennec Native with off-main-thread compositing, so we should indeed uplift this when off-main-thread compositing gets uplifted.

Comment 17

[Scoobidiver (away)]

(In reply to Ali Juma [:ajuma] from comment #16)
> Bug 694964 was backed out of FF10 on Jan. 23 (see Comment 2 above), so this
> crash shouldn't be happening there.

Comment 18

[Alex Keybl [:akeybl]]

No need to track for FF13 at this point - we don't expect to ship a Fennec build off of that version.

Comment 19

[Robert Kaiser]

"TouchBadMemory | mozalloc_abort | nsHtml5ElementName::releaseStatics" is currently roughly half of our Fennec 13 Beta crashes, if this is this bug, is there any chance to get the fix uplifted to beta?