1114477 - [Gonk-L-Camera]:Integrate Camera recording.

Status: RESOLVED FIXED

(Firefox OS Graveyard :: GonkIntegration, defect)

Description

[Vincent Liu[:vliu]]

This is a porting issue based on bug 1107300 Comment 17.

Comment 1

[Vincent Liu[:vliu]]

Attachment: 5.log

I got some error messages when I tried to do camera recording. The detailed log print was attached. It seems that these messages cause OMX Component deinit.
Does anyone can help me to figure it out? Thanks.

E/OMXMaster(  186): A component of name 'OMX.qcom.audio.decoder.aac' already exists, ignoring this one.
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
D/camera  ( 1244): OnNewPreviewFrame: we have 1 preview frame listener(s)
D/camera  ( 1244): OnNewPreviewFrame: got 1920 x 1080 frame
D/GonkNativeWindow( 1244): GonkNativeWindow::returnBuffer
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663311) ERROR: 0x8000101a
E/OMXNodeInstance(  186): getParameter(100663297) ERROR: 0x8000101a

These error codes were indicating OMX_ErrorUnsupportedIndex.

Comment 2

[Vincent Liu[:vliu]]

Attachment: gdb-bt.log

gdb backtrack for the first error code which OMXCodec was trying to QueryCodec.

Comment 3

[Vincent Liu[:vliu]]

Hi Blake,

Would you please have some idea for the log I attached? Thanks.

Comment 4

[Blake Wu [:bwu][:blakewu]]

Comment on attachment 8539970 [details]
gdb-bt.log

John should be better than me to answer this question. :)

Comment 5

[John Lin [:jhlin][:jolin]]

According to the backtrace in gdb-bt.log, it's querying audio codec (mime type "audio/mpeg"), but the index to query is for  video codec only (IndexParamVideoProfileLevelQuerySupported).
FWICT, QueryCodec() returns OK (with empty CodecCapabilities) and MediaCodecList just pass that to MediaCodecInfo. It seems those are not actual errors in this case.

Comment 6

[Vincent Liu[:vliu]]

Attachment: camera-rec-7.log

Hi John,

I tried to enable the log flag for OMXCodec/ACodec and make sure the library has beed updated by date. The file was attached. Could you please also have a check for me? Thanks

Comment 7

[Steven Yang [:styang]]

It's the porting issue. Marked feature-b2g : 2.2+.

Hi Vincent, we will need ETA for better understanding the risk to the next sprint. Thanks.

Comment 8

[John Lin [:jhlin][:jolin]]

Tried adding some code to dump data in the AVC/H.264 buffers from encoder and found it playable in VLC. I guess the problem is not the encoder.

Comment 9

[John Lin [:jhlin][:jolin]]

Attachment: ** NOT FOR CHECKING IN** patch to make stagefright log appear.

A hacky patch to work around the 'stagefright log not showing up' problem (It's because of failed socket() call at [1], but I don't know why that happens). Anyway this should make debugging much easier.

[1] https://android.googlesource.com/platform/system/core/+/master/liblog/logd_write.c#113

Comment 10

[Steven Yang [:styang]]

Hi John, assign to you since you're working on it.

Comment 11

[John Lin [:jhlin][:jolin]]

(In reply to Steven Yang [:styang] from comment #10)
> Hi John, assign to you since you're working on it.

Actually Vincent & Becker are working very hard on this one, but I don't mind taking the credit. :P

Comment 12

[Blake Wu [:bwu][:blakewu]]

(In reply to John Lin [:jolin][:jhlin] from comment #9)
> Created attachment 8541687 [details] [diff] [review]
> ** NOT FOR CHECKING IN** patch to make stagefright log appear.
> 
> A hacky patch to work around the 'stagefright log not showing up' problem
> (It's because of failed socket() call at [1], but I don't know why that
> happens). Anyway this should make debugging much easier.
> 
> [1]
> https://android.googlesource.com/platform/system/core/+/master/liblog/
> logd_write.c#113
Maybe we should create a bug for this logging problem.

Comment 13

[Vincent Liu[:vliu]]

When stoping the recording, video track would block at pthread_join().

https://github.com/mozilla-b2g/platform_frameworks_av/blob/b2g-5.0.0_r6/media/libstagefright/MPEG4Writer.cpp#L1791

The root casue would be blocking in the below wait function.

https://github.com/mozilla-b2g/platform_frameworks_av/blob/b2g-5.0.0_r6/media/libstagefright/OMXCodec.cpp#L3999

Here has a considered solution. 

https://android.googlesource.com/platform/frameworks/av/+/bf37f33/media/libstagefright/MPEG4Writer.cpp

The main idea is switching the calling order for |pthread_join(mThread, &dummy)| and |mSource->stop()|. When mDone turns to true, making sure the thread is done from exiting pthread_join and then calling mSource->stop() to stop the data feeding. Flame-kk also run on this order.

Comment 14

[John Lin [:jhlin][:jolin]]

(In reply to Vincent Liu[:vliu] from comment #13)
> When stoping the recording, video track would block at pthread_join().
> 
> https://github.com/mozilla-b2g/platform_frameworks_av/blob/b2g-5.0.0_r6/
> media/libstagefright/MPEG4Writer.cpp#L1791
> 
> The root casue would be blocking in the below wait function.
> 
> https://github.com/mozilla-b2g/platform_frameworks_av/blob/b2g-5.0.0_r6/
> media/libstagefright/OMXCodec.cpp#L3999
> 
> Here has a considered solution. 
> 
> https://android.googlesource.com/platform/frameworks/av/+/bf37f33/media/
> libstagefright/MPEG4Writer.cpp
> 
> The main idea is switching the calling order for |pthread_join(mThread,
> &dummy)| and |mSource->stop()|. When mDone turns to true, making sure the
> thread is done from exiting pthread_join and then calling mSource->stop() to
> stop the data feeding. Flame-kk also run on this order.

 The proposed fix essentially undo what Google did in [1]. From the commit message it seems to me that the change is part of rewriting Android recorder implementation to use MediaCodec (and deprecate OMXCodec). I'll test if the fix breaks Android recording functionality. If not then this could be a low risk solution for us.

 [1] https://android.googlesource.com/platform/frameworks/av/+/72cecca17d735db6532c45f0a7e10c47ee6f065a

Comment 15

[John Lin [:jhlin][:jolin]]

Change component to gonk integration, where the fix will be more likely to land.

Comment 16

[John Lin [:jhlin][:jolin]]

Attachment: Don't stop source until track thread is stopped.

Hi Mike, could you please help review this PR? I tested it on AOSP as mentioned in comment 14 and it seems no problem. Thanks a lot & happy new year.

Comment 17

[Mike Habicher [:mikeh] (high bugzilla latency)]

Comment on attachment 8543814 [details] [review]
Don't stop source until track thread is stopped.

LGTM.

Comment 18

[Ryan VanderMeulen [:RyanVM]]

b2g-5.0.0_r6: https://github.com/mozilla-b2g/platform_frameworks_av/commit/8642b85281d90c0d804c382a884ebd1c113b9570

Comment 19

[Hermes Cheng (inactive after July 27, 2015)]

Please also land the patch to v2.2 branch, thanks.