995476 - crash in __libc_android_abort | __android_log_assert | android::OMXCodec::~OMXCodec

Status: RESOLVED DUPLICATE of bug 977797

(Core :: Audio/Video, defect)

Description

[Rachel Pribble [:rpribble]]

Attachment: Dmesg.txt

This bug was filed from the Socorro interface and is 
report bp-dc98f799-2412-4bef-aa25-1de192140411.
=============================================================

Description:
(Expand upon the Summary - but not a copy of the Summary!)

Repro Steps:
1) Updated Buri to BuildID: 20140411000202
2) Tap email icon
3) Setup email
4) Create new email with video attachment
5) Observe crash while video selection screen is loading

Actual:
Crash during video selection screen.

Expected:
User able to attach videos to emails effectively.

Environmental Variables:
Device: Buri v1.4 MOZ ril
BuildID: 20140411000202
Gaia: 6c50349f41d40ba175ea0fc0c2c2cbd739ba7170
Gecko: 28b419f0e857
Version: 30.0a2
Firmware Version: v1.2-device.cfg

Notes:

Repro frequency: 3/3, 100%
See attached: Video, dmesg, logcat

Comment 1

[Rachel Pribble [:rpribble]]

Attachment: Logcat.txt

This issue also occurs on the 3/26/14 Buri v1.4 MOZ ril (before the new camera).

Environmental Variables:
Device: Buri v1.4 MOZ ril
BuildID: 20140326000201
Gaia: 7e705dd4718d528974d99ac31866318d7e201152
Gecko: 4889124accfa
Version: 30.0a2
Firmware Version: v1.2-device.cfg

Crash occurs when video selection screen is loading.

Comment 2

[Rachel Pribble [:rpribble]]

This issue does not occur on the Buri v1.3 MOZ ril.

Environmental Variables:
Device: Buri v1.3 MOZ ril
BuildID: 20140410004002
Gaia: 62acb4b0e774b6709b8be400d849f807404bb21b
Gecko: 94baf6039462
Version: 28.0
Firmware Version: v1.2-device.cfg

Video selection screen loads as expected.

Comment 3

[Sotaro Ikeda [:sotaro]]

(In reply to Rachel Pribble from comment #1)
> Created attachment 8405653 [details]
> Logcat.txt
> 
> This issue also occurs on the 3/26/14 Buri v1.4 MOZ ril (before the new
> camera).
> 
> Environmental Variables:
> Device: Buri v1.4 MOZ ril
> BuildID: 20140326000201
> Gaia: 7e705dd4718d528974d99ac31866318d7e201152
> Gecko: 4889124accfa
> Version: 30.0a2
> Firmware Version: v1.2-device.cfg
> 
> Crash occurs when video selection screen is loading.


out of pmem seems to cause the crash.

04-11 19:04:53.909: E/memalloc(136): /dev/pmem_adsp: failed to map pmem fd: Out of memory
04-11 19:04:53.909: E/memalloc(136): virtual int gralloc::PmemAshmemController::allocate(gralloc::alloc_data&, int, int): Failed to allocate ADSP/SMI memory
04-11 19:04:53.909: E/msm7627a.gralloc(136): gralloc failed err=Out of memory
04-11 19:04:53.909: W/GraphicBufferAllocator(136): alloc(720, 480, 138, 10102100, ...) failed -12 (Out of memory)
04-11 19:04:53.909: I/Gecko(1519): dequeueBuffer: failed to alloc gralloc buffer
04-11 19:04:53.909: E/OMXCodec(1519): dequeueBuffer failed: Out of memory (12)
04-11 19:04:53.909: E/OMXCodec(1519): [OMX.qcom.video.decoder.avc] Allocate Buffer failed - error = -12
04-11 19:04:53.919: A/OMXCodec(1519): frameworks/base/media/libstagefright/OMXCodec.cpp:2307 CHECK(mState == LOADED || mState == ERROR || mState == LOADED_TO_IDLE) failed.

Comment 4

[Sotaro Ikeda [:sotaro]]

The following log says, failed grallc buffer's size is (with=720, height = 480). Which sizes of videos did you use for test?

> W/GraphicBufferAllocator(136): alloc(720, 480, 138, 10102100, ...) failed -12 (Out of memory)

Comment 5

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #4)
> The following log says, failed grallc buffer's size is (with=720, height =
> 480). Which sizes of videos did you use for test?
> 
> > W/GraphicBufferAllocator(136): alloc(720, 480, 138, 10102100, ...) failed -12 (Out of memory)

Rachel, an additional question. Is there no problem just open video app?

Comment 6

[Rachel Pribble [:rpribble]]

Opening the video app does result in a crash, but the crash report links to bug 977797 rather than the one encountered in this thread.

Environmental Variables:
Device: Buri v1.4 MOZ ril
BuildID: 20140414000201
Gaia: 8dff633372022723e2ebad17fe3c826436b3b258
Gecko: bc14179fc49c
Version: 30.0a2
Firmware Version: v1.2-device.cfg

Comment 7

[Rachel Pribble [:rpribble]]

Actually, after a few more attempts another crash report links back to this bug (995476), as well.

Comment 8

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #5)
> (In reply to Sotaro Ikeda [:sotaro] from comment #4)
> > The following log says, failed grallc buffer's size is (with=720, height =
> > 480). Which sizes of videos did you use for test?

Rachel, can you answer the above question?

Comment 9

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #4)
> The following log says, failed grallc buffer's size is (with=720, height =
> 480). Which sizes of videos did you use for test?
> 
> > W/GraphicBufferAllocator(136): alloc(720, 480, 138, 10102100, ...) failed -12 (Out of memory)

Different crash, but it seems dup of Bug 977797. See Bug 977797 Comment 7.

Comment 10

[Rachel Pribble [:rpribble]]

Sorry about that. On my device, I have around 15 videos taken manually ranging in size from 1.3 MB - 32 MB. A few of the videos were taken on other devices with my sim.

Comment 11

[Kai-Chih Hu [:khu]]

(In reply to Rachel Pribble from comment #10)
> Sorry about that. On my device, I have around 15 videos taken manually
> ranging in size from 1.3 MB - 32 MB. A few of the videos were taken on other
> devices with my sim.

So, did you mean that, no matter what video files you tried, it always crashed? 
And, can you play these video files without any problem?

Comment 12

[Rachel Pribble [:rpribble]]

Yes, crash is occurring regardless of what videos are used. I removed over half of the videos from the sd card through usb sharing and the crash still occurs.  I am currently unable to play videos successfully as the video app is crashing consistently with the report linking to bug 977797. 

Environmental Variables:
Device: Buri v1.4 MOZ ril
BuildID: 20140417000204
Gaia: 00a9d55a3e7463cecfb5dde185c0ee1f4c4d9e54
Gecko: d3d40652aaa2
Version: 30.0a2
Firmware Version: v1.2-device.cfg

Comment 13

[Rachel Pribble [:rpribble]]

The crash appears to be caused by a memory issue, it seems easier to reproduce with larger-sized videos. 

I've tried several different file types and no file type in particular causes a crash. 

During transferring a batch of files on to the device, when the warning prompt 'not enough space' is seen on the computer, I removed one individual file so that the transfer would fit. The crash was then seen. 

It appears that the user sees the crash when the device is near capacity.

Comment 14

[Harvey]

The pushlog indicates a merge from mozilla-inbound to m-c, unfortunately we do not have mozilla-inbound builds available in this date range to provide a deeper window.

Tinderbox Regression Window

Last Working Environmental Variables:
Device: Buri
BuildID: 20140214054315
Gaia: ade88673d00414c3177f7444543b2fa01324708e
Gecko: 23f7a629a217
Version: 30.0a1
Firmware Version: v1.2-device.cfg

First Broken Environmental Variables:
Device: Buri
BuildID: 20140214055415
Gaia: ade88673d00414c3177f7444543b2fa01324708e
Gecko: 5d7caa093f4f
Version: 30.0a1
Firmware Version: v1.2-device.cfg

Gaia is the same on both builds pointing to this being a Gecko issue.

Gecko Pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=23f7a629a217&tochange=5d7caa093f4f

Comment 15

[Jason Smith [:jsmith]]

Eric - Can you get someone to look at this?

Comment 16

[Kai-Chih Hu [:khu]]

Will try to find an owner for this bug.

Comment 17

[Eric Chou [:ericchou] [:echou]]

(In reply to Jason Smith [:jsmith] from comment #15)
> Eric - Can you get someone to look at this?

Sorry but I don't think Blake and Bruce have enough bandwidth to look into this one ... I know Sotaro is an expert of Android OMXCodec. Maybe he could help.

Hi Sorato,

Would you mind taking a first look? Please don't hesitate to tell me if you're not available. Thanks!

Comment 18

[Sotaro Ikeda [:sotaro]]

> 
> Hi Sorato,
> 
> Would you mind taking a first look? Please don't hesitate to tell me if
> you're not available. Thanks!

Hi Eric, I could have a bandwidth from next Wednesday. Is it OK?

Comment 19

[Sotaro Ikeda [:sotaro]]

Rachel, can I have the content that caused the crash?

Comment 20

[Eric Chou [:ericchou] [:echou]]

(In reply to Sotaro Ikeda [:sotaro] from comment #18)
> > 
> > Hi Sorato,
> > 
> > Would you mind taking a first look? Please don't hesitate to tell me if
> > you're not available. Thanks!
> 
> Hi Eric, I could have a bandwidth from next Wednesday. Is it OK?

I think it should be ok. ni? Kevin Hu, one of 1.4 release owners, to double confirm.

Comment 21

[Kai-Chih Hu [:khu]]

Looks like there has no choice. I think I am fine with it. If there has any better solutions, please let me know. Thanks. :-)

Comment 22

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #19)
> Rachel, can I have the content that caused the crash?

It seems that attachment 8383247 [details] in Bug 977797 can be used for the test. Thanks.

Comment 23

[Sotaro Ikeda [:sotaro]]

master's error handling is going to be replaced by Bug 959089. This bug is necessary only for v1.4.

Comment 24

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #23)
> master's error handling is going to be replaced by Bug 959089. This bug is
> necessary only for v1.4.

Sorry, the above comment is for another bug.

Comment 25

[Sotaro Ikeda [:sotaro]]

I compared logcat log of v1.3 hamachi and master hamachi. Even when OMXCodec is same, the logout of OMXCodec is different. From it, functions call timings from gecko seems different.

Comment 26

[Sotaro Ikeda [:sotaro]]

When video app is opened with attachment 8383247 [details] in sd card, Bug 977797's crash happened. From the crash log, MediaBuffer::release() seems already deleted.

Comment 27

[Sotaro Ikeda [:sotaro]]

Attachment: logcat - MediaBuffer::release() crash

Crash log that mentioned in Comment 26.

Comment 28

[Sotaro Ikeda [:sotaro]]

attachment 8383247 [details] can not be decoded both v1.3 hamachi and master hamachi. But there is a difference in the logcat. on v1.3 hamachi, failure happened only at fillBuffer. But on master hamachi, empty buffer is also failed and did freeing input buffer.

> E OMXCodec: [OMX.qcom.video.decoder.avc] fillBuffer failed w/ error 0x80000000
> V OMXCodec: [OMX.qcom.video.decoder.avc] Calling lockBuffer on 0x1b11eb0
> V OMXCodec: [OMX.qcom.video.decoder.avc] Calling fillBuffer on buffer 0x1b11eb0
> E omx_vdec: FTB in Invalid State
> E OMXCodec: [OMX.qcom.video.decoder.avc] fillBuffer failed w/ error 0x80000000
> V OMXCodec: [OMX.qcom.video.decoder.avc] Calling lockBuffer on 0x1b11f00
> V OMXCodec: [OMX.qcom.video.decoder.avc] Calling fillBuffer on buffer 0x1b11f00
> E omx_vdec: FTB in Invalid State
> E OMXCodec: [OMX.qcom.video.decoder.avc] fillBuffer failed w/ error 0x80000000
> V OMXCodec: [OMX.qcom.video.decoder.avc] EMPTY_BUFFER_DONE(buffer: 0x1b05c58)
> E OMXCodec: [OMX.qcom.video.decoder.avc] ERROR(0x8000100a, 0)
> V OMXCodec: [OMX.qcom.video.decoder.avc] EMPTY_BUFFER_DONE(buffer: 0x1b05ca8)
> V OMXCodec: [OMX.qcom.video.decoder.avc] mState ERROR, freeing i/p buffer 0x1b05ca8
> V OMXCodec: [OMX.qcom.video.decoder.avc] pause mState=11


v1.3 hamachi

> E/omx_vdec(  140): FTB in Invalid State
> E/OMXCodec(  517): [OMX.qcom.video.decoder.avc] fillBuffer failed w/ error 0x80000000
> E/omx_vdec(  140): FTB in Invalid State
> E/OMXCodec(  517): [OMX.qcom.video.decoder.avc] fillBuffer failed w/ error 0x80000000
> E/omx_vdec(  140): FTB in Invalid State
> E/OMXCodec(  517): [OMX.qcom.video.decoder.avc] fillBuffer failed w/ error 0x80000000
> E/omx_vdec(  140): FTB in Invalid State
> E/OMXCodec(  517): [OMX.qcom.video.decoder.avc] fillBuffer failed w/ error 0x80000000
> E/OMXCodec(  517): [OMX.qcom.video.decoder.avc] ERROR(0x8000100a, 0)
> E/OMXCodec(  517): [OMX.qcom.video.decoder.avc] ERROR(0x8000100a, 0)
> E/OMXCodec(  517): [OMX.qcom.video.decoder.avc] in error state, check omx il state and decide whether to free or skip
> E/OMXCodec(  517): [OMX.qcom.video.decoder.avc] OMX IL is in state 0

Comment 29

[Sotaro Ikeda [:sotaro]]

"mState ERROR, freeing i/p buffer" is a different part compared to v1.3 hamachi. The log is printed in the following code. The code free the input buffer. This code is not present in AOSP. It seems that the code is added by caf.

void OMXCodec::on_message(const omx_message &msg) {
>>snip
>            BufferInfo* info = &buffers->editItemAt(i);
>            info->mStatus = OWNED_BY_US;
>            if ((mState == ERROR)  && (bInvalidState == true)) {
>              CODEC_LOGV("mState ERROR, freeing i/p buffer %p", buffer);
>              status_t err = freeBuffer(kPortIndexInput, i);
>              CHECK_EQ(err, (status_t)OK);
>            }

Comment 30

[Sotaro Ikeda [:sotaro]]

Attachment: patch - remove input buffer error handling

I confirmed that by applying the patch, the problem is fixed on master hamachi.

Comment 31

[Sotaro Ikeda [:sotaro]]

Inder, can I have a comment to attachment 8414048 [details] [diff] [review]?

Comment 32

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #31)
> Inder, can I have a comment to attachment 8414048 [details] [diff] [review]?

By applying attachment 8414048 [details] [diff] [review], Bug 977797 also fixed. Bug 977797 seems dup of this bug.

Comment 33

[Sotaro Ikeda [:sotaro]]

This bug seems OMXCodec's problem. Therefore it seems POVB problem.

Comment 34

[Eric Chou [:ericchou] [:echou]]

(In reply to Sotaro Ikeda [:sotaro] from comment #33)
> This bug seems OMXCodec's problem. Therefore it seems POVB problem.

Per comment 33, set [POVB] for partner to track. Thanks, Sotaro.

Comment 35

[Jason Smith [:jsmith]]

(In reply to Eric Chou [:ericchou] [:echou] from comment #34)
> (In reply to Sotaro Ikeda [:sotaro] from comment #33)
> > This bug seems OMXCodec's problem. Therefore it seems POVB problem.
> 
> Per comment 33, set [POVB] for partner to track. Thanks, Sotaro.

I don't agree this is a vendor problem. If something bad happens on the vendor side, then we should be defensive against it to make sure that we don't crash. That is what is needed to be fixed here.

Comment 36

[Eric Chou [:ericchou] [:echou]]

(In reply to Jason Smith [:jsmith] from comment #35)
> (In reply to Eric Chou [:ericchou] [:echou] from comment #34)
> > (In reply to Sotaro Ikeda [:sotaro] from comment #33)
> > > This bug seems OMXCodec's problem. Therefore it seems POVB problem.
> > 
> > Per comment 33, set [POVB] for partner to track. Thanks, Sotaro.
> 
> I don't agree this is a vendor problem. If something bad happens on the
> vendor side, then we should be defensive against it to make sure that we
> don't crash. That is what is needed to be fixed here.

I totally agree but that's not the case: the crash did happen at caf's code. We use the library which is fetched from caf and the process crashes inside. I can't think of a better solution to fix this on our side if the library just crashed itself. Please see comment 29 for more information.

Comment 37

[Jason Smith [:jsmith]]

Ok - seeing Sotaro's comment above, this is the same problem as bug 977797, so I'm duping over to that bug.

Comment 38

[Sotaro Ikeda [:sotaro]]

(In reply to Jason Smith [:jsmith] from comment #35)
> (In reply to Eric Chou [:ericchou] [:echou] from comment #34)
> > (In reply to Sotaro Ikeda [:sotaro] from comment #33)
> > > This bug seems OMXCodec's problem. Therefore it seems POVB problem.
> > 
> > Per comment 33, set [POVB] for partner to track. Thanks, Sotaro.
> 
> I don't agree this is a vendor problem. If something bad happens on the
> vendor side, then we should be defensive against it to make sure that we
> don't crash. That is what is needed to be fixed here.

I do not thinks we can protect from this OMXCodec's defect.

Comment 39

[Inder]

(In reply to Sotaro Ikeda [:sotaro] from comment #31)
> Inder, can I have a comment to attachment 8414048 [details] [diff] [review]?

Sotaro -- not really sure but it's possible this is an issue in ICS version of OMXCodec. I see this code removed in JB and KK version of OMXCodec.

Comment 40

[Sotaro Ikeda [:sotaro]]

(In reply to Inder from comment #39)
> (In reply to Sotaro Ikeda [:sotaro] from comment #31)
> > Inder, can I have a comment to attachment 8414048 [details] [diff] [review]?
> 
> Sotaro -- not really sure but it's possible this is an issue in ICS version
> of OMXCodec. I see this code removed in JB and KK version of OMXCodec.

Thanks for the comment. I think that the code causes the problem when EMPTY_BUFFER_DONE returns error. And this error happens only when kernel(hw codec) config is not correct. hamachi(v1.2-device.cfg rom) seems to have a problem to the config. See Bug 977797 Comment 16.