932204 - [flatfish][camera] After pressing the recording button, the icon of camera switch is disappeared

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Gaia::Camera, defect)

Description

[JustinLee]

Steps to reproduce:
1. Open Camera app
2. Switch to video recording mode
3. Press "video recording" button => pop-up a message "Video not recorded. An error..... the video."
4. Press "OK"

Actual results:
The icon of camera switch is disappeared. The user can't switch front/rear camera.

Comment 1

[Vincent Liu[:vliu]]

Currently narrow down the fail point in 

http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkRecorder.cpp#1269

I still figure out why it fails when it called.

Comment 2

[Vincent Liu[:vliu]]

OMXCodec::Create() failed because encoder_flags always be 0. Backed trace the code and currently narrow down the fail point in  

http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkCameraSource.cpp#482

Comment 3

[saminath]

This is due to Video Recording patches are not integrated in the Main-stream line.
Could you integrate patches(4) from the below link and test it.
https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c31

Initailly I faced same, after this patch for Video recording integration, start working for me.

Comment 4

[Keven Kuo [:kkuo]]

Hi! Vincent,

Please give it a try. Thanks.

--
Kwvwn

Comment 5

[Vincent Liu[:vliu]]

(In reply to saminath from comment #3)
> This is due to Video Recording patches are not integrated in the Main-stream
> line.
> Could you integrate patches(4) from the below link and test it.
> https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c31
> 
> Initailly I faced same, after this patch for Video recording integration,
> start working for me.

I tried you suggestion but still fail. May I confirm your steps.

1. Which device you used in your test?
2. In the patch you showed to me, it only change b2g.sh to use a umask of 0007 instead of 0027, right?

Thanks.

Comment 6

[saminath]

Hi Vincent.

1. Which device you used in your test?
I tested on Nexus-4 device

2. In the patch you showed to me, it only change b2g.sh to use a umask of 0007 instead of 0027, right?

No. Please do integration all 4 Patches in the attachment (0001-HACK-Change-umask-to-007.patch,0001-HACK-Mount-internal-sdcard-to-data-sdcard-and-change.patch,bug-910498-getVol-n4.patch,bug-910498-hack.patch).

Thanks.

Comment 7

[Kai-Chih Hu [:khu]]

Taipei triage result: ---.

Comment 8

[RaymondSu]

This is functional fail of Flatfish camera.  Hi, Francis: please confirm.

Comment 9

[Francis Lee [:frlee]]

for comment #7, the reason is: until we have final decision of Flatfish base version (v1.2 or v1.3), change to --- for now.

for comment #8, Raymond, this is an issue we must fix.

Comment 10

[Vincent Liu[:vliu]]

(In reply to saminath from comment #6)
> Hi Vincent.
> 
> 1. Which device you used in your test?
> I tested on Nexus-4 device
> 
> 2. In the patch you showed to me, it only change b2g.sh to use a umask of
> 0007 instead of 0027, right?
> 
> No. Please do integration all 4 Patches in the attachment
> (0001-HACK-Change-umask-to-007.patch,0001-HACK-Mount-internal-sdcard-to-data-
> sdcard-and-change.patch,bug-910498-getVol-n4.patch,bug-910498-hack.patch).
> 
> Thanks.

As I observed on Flatfish, it is not the same case I saw on Nexus-4. Nexus-4 get fails on the blow link.

http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkCameraControl.cpp#1087

But for Flatfish, it didn't block on GetVolumeByPath(..). Please see the comment 2 to know the actual block point. Thanks.

Comment 11

[Francis Lee [:frlee]]

as we confirmed that Flatfish will use v1.3 and we need to respect current testing cycle, set target milestone to 12/6 since 1.3FC tag is not created yet.

Comment 12

[Vincent Liu[:vliu]]

(In reply to Vincent Liu[:vliu] from comment #2)
> OMXCodec::Create() failed because encoder_flags always be 0. Backed trace
> the code and currently narrow down the fail point in  
> 
> http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkCameraSource.
> cpp#482

Hi JustinLee,

I traced the code and it failed in the link in Comment 2. |mCameraHw->StoreMetaDataInBuffers(true)| binds to libcameraservice in mediaserver and then calls to CameraHardware. Can you also find this case? I need your help to figure out why it returns fail. Thanks

Comment 13

[Vincent Liu[:vliu]]

Attachment: WIP-1.txt

The attached file is a WIP. With this patch, camera recording won't be blocked by a gaia message "video not recorded..." and has a recording status. But when I tried to press the button to stop the recording, it won't store any video file. For this, I still look into it.

Comment 14

[JustinLee]

Hi Vincent,

I traced the code and found that the return value of storeMetaDataInBuffers(bool enable) is UNKNOWN_ERROR only in camera HAL on flatfish.
FYR.

Comment 15

[Vincent Liu[:vliu]]

When Camera app start video recording, OMXCodec received a message with |event = OMX_EventError|

https://android.googlesource.com/platform/frameworks/av/+/android-4.2.2_r1.2/media/libstagefright/OMXCodec.cpp

The detailed message showed below

(gdb) p msg
$3 = (const android::omx_message &) @0x428fdd58: {type = android::omx_message::EVENT, node = 0x1, u = {
    event_data = {event = OMX_EventError, data1 = 2147487744, data2 = 0}, buffer_data = {buffer = 0x1}, 
    extended_buffer_data = {buffer = 0x1, range_offset = 2147487744, range_length = 0, flags = 0, timestamp = 
    -5527349839097065472, platform_private = 0x42e77a48, data_ptr = 0x0}}}

I need help from partner to look into why OMX_EventError is received by OMXCodec. Thanks

Comment 16

[JustinLee]

OMX_EventError data1 is 214748774 (in Dec., the HEX is 0x80001000), the error means OMX_ErrorInsufficientResources. I think that video format may be wrong.

In addition, camera app in gaia uses 3gpp as default video format (Below shows the partial code of camera.js). The video format of Android camera app is mp4 on flatfish platform.

http://mxr.mozilla.org/gaia/source/apps/camera/js/camera.js#31
var dcfConfig = {
    key: 'dcf_key',
    seq: null,
    postFix: 'MZLLA',
    prefix: {video: 'VID_', image: 'IMG_'},
    ext: {video: '3gp', image: 'jpg'}
  };

http://mxr.mozilla.org/gaia/source/apps/camera/js/camera.js#658
var dummyblob = new Blob([''], {type: 'video/3gpp'});

Video format may be a key point on video recording.


BTW, the video format of video recording on onetouchFire is 3gpp. Dose it use 3gpp software encode?

Comment 17

[Vincent Liu[:vliu]]

(In reply to JustinLee from comment #16)
> OMX_EventError data1 is 214748774 (in Dec., the HEX is 0x80001000), the
> error means OMX_ErrorInsufficientResources. I think that video format may be
> wrong.
> 
> In addition, camera app in gaia uses 3gpp as default video format (Below
> shows the partial code of camera.js). The video format of Android camera app
> is mp4 on flatfish platform.
> 
> http://mxr.mozilla.org/gaia/source/apps/camera/js/camera.js#31
> var dcfConfig = {
>     key: 'dcf_key',
>     seq: null,
>     postFix: 'MZLLA',
>     prefix: {video: 'VID_', image: 'IMG_'},
>     ext: {video: '3gp', image: 'jpg'}
>   };
> 
> http://mxr.mozilla.org/gaia/source/apps/camera/js/camera.js#658
> var dummyblob = new Blob([''], {type: 'video/3gpp'});
> 
> Video format may be a key point on video recording.
> 
> 
> BTW, the video format of video recording on onetouchFire is 3gpp. Dose it
> use 3gpp software encode?

Gaia createed a dummyblob by assigning ext and type for a container. The actual output format for video encoding for a container is retrieved from below

http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkRecorderProfiles.cpp#80

If you dump mPlatformOutputFormat, you will get OUTPUT_FORMAT_MPEG_4.

Comment 18

[Vincent Liu[:vliu]]

Currently the error state happens because Codec's input buffers are too small to accomodate buffer read from source. Please see the below code snippet in frameworks/av/media/libstagefright/OMXCodec.cpp.

OMXCodec::drainInputBuffer(BufferInfo *info) {

        if (srcBuffer->range_length() > remainingBytes) {
            if (offset == 0) {
                ALOGE(
                     " Codec's input buffers are too small to accomodate "
                     "buffer read from source (info->mSize = %d, srcLength = %d)",
                     info->mSize, srcBuffer->range_length());
                
                srcBuffer->release();
                srcBuffer = NULL;

                setState(ERROR);
                return false;
            }
}

I will then focus on how to determine the buffer size allocation for both codec input buffer and source buffer. Please feel free to let me know relative information if you well known about this. Thanks.

Comment 19

[Vincent Liu[:vliu]]

The value of info->mSize is assigned from calling the OMXCodec::setVideoInputFormat(...). Compared with Nexus-4 and the value seems normal. But for srcBuffer->range_length(), the value is much larger then Nexus-4. The value of srcBuffer->range_length() is assigned by calling the GonkCameraSource::dataCallbackTimestamp(...). This callback function is wrapped by CameraHardware in camera.flatfish.so through HAL. 

#0  android::CameraClient::dataCallbackTimestamp (timestamp=<optimized out>, msgType=32, 
    dataPtr=..., user=0x0)
    at frameworks/av/services/camera/libcameraservice/CameraClient.cpp:753
#1  0x404ffbd8 in android::CameraHardwareInterface::__data_cb_timestamp (
    timestamp=<optimized out>, msg_type=32, data=<optimized out>, index=0, user=0x416e6788)
    at frameworks/av/services/camera/libcameraservice/CameraHardwareInterface.h:484
#2  0x40a6f65e in android::CallbackNotifier::onNextFrameSW(void const*) ()
   from /home/vliu-laptop/projs/a31-422/out/target/product/flatfish/system/lib/hw/camera.flatfish.so
#3  0x40a75a68 in android::V4L2CameraDevice::previewThread() ()
   from /home/vliu-laptop/projs/a31-422/out/target/product/flatfish/system/lib/hw/camera.flatfish.so

I think the next move should focus on fr 2.

Comment 21

[Vincent Liu[:vliu]]

Attachment: WIP.patch

Here to summarize some issues we'd met for this issue.

1. AAC recording was not support on JB 4.2.2 base. Bug 944567 was filed for this issue tracking.
2. mCameraHw->StoreMetaDataInBuffers(true) always returns UNKNOWN_ERROR.
   Action : Partner modified code in camera.flatfish.so to force to return NO_ERROR. They will land to bitbucket server.
3. Codec's Input buffer size was smaller than Source Buffer Size.
   Cause : Camera hardware read 1280X720 as its default resolution. It conflict the the resolution we set from Gecko.
   Action : Modify the resolution from camera.cfg to match to the same with Gecko.
4. Frame rate conflicted for front camera.
   Cause : Gecko read unsupported frame rate and indicated error recording.
   Action : Modified supported setting in camera.cfg
5. SampleDelta in the first stts table was was much larger than expected value.
   Cause 1 : The unit of timestamp Gecko and camera hardware looked different. The unit of timestamp camera hardware was 1000 times than it looked in Gecko.
   Action 1 : A work around modified in GonkCameraSource.cpp. Please see WIP.patch to know the detail.
              Currently it only happens in Flatfish platform. 

   Cause 2 : The first SampleDelta of stts table was much larger than expected value. 
   Action 2 : A work around modified in MPEG4Writer.cpp. Please see WIP.patch to know the detail.
              
   Actually the timestamp from camera hardware is quite larger than we saw from Nexus-4. As my point of view, it should be the root cause for item 5. After discussed with partner, they will keep working on it.

Comment 22

[JustinLee]

(In reply to Vincent Liu[:vliu] from comment #21)
> Created attachment 8345644 [details] [diff] [review]
> WIP.patch
> 
> Here to summarize some issues we'd met for this issue.
> 
> 1. AAC recording was not support on JB 4.2.2 base. Bug 944567 was filed for
> this issue tracking.
> 2. mCameraHw->StoreMetaDataInBuffers(true) always returns UNKNOWN_ERROR.
>    Action : Partner modified code in camera.flatfish.so to force to return
> NO_ERROR. They will land to bitbucket server.
> 3. Codec's Input buffer size was smaller than Source Buffer Size.
>    Cause : Camera hardware read 1280X720 as its default resolution. It
> conflict the the resolution we set from Gecko.
>    Action : Modify the resolution from camera.cfg to match to the same with
> Gecko.
> 4. Frame rate conflicted for front camera.
>    Cause : Gecko read unsupported frame rate and indicated error recording.
>    Action : Modified supported setting in camera.cfg
> 5. SampleDelta in the first stts table was was much larger than expected
> value.
>    Cause 1 : The unit of timestamp Gecko and camera hardware looked
> different. The unit of timestamp camera hardware was 1000 times than it
> looked in Gecko.
>    Action 1 : A work around modified in GonkCameraSource.cpp. Please see
> WIP.patch to know the detail.
>               Currently it only happens in Flatfish platform. 
> 
>    Cause 2 : The first SampleDelta of stts table was much larger than
> expected value. 
>    Action 2 : A work around modified in MPEG4Writer.cpp. Please see
> WIP.patch to know the detail.
>               
>    Actually the timestamp from camera hardware is quite larger than we saw
> from Nexus-4. As my point of view, it should be the root cause for item 5.
> After discussed with partner, they will keep working on it.

About item 5, we have modified camera HAL for fixing timestamp issue. 
So far, camera recording don't need WIP.patch (attachment #8345644 [details] [diff] [review]) on flatfish platform.

Comment 23

[Jason Smith [:jsmith]]

I'm unsure why this would be 1.3 blocker specifically, as I don't recall work happening for flatfish on any branch other than trunk. Renominating for more discussion.

Comment 24

[Jason Smith [:jsmith]]

See https://bugzilla.mozilla.org/show_bug.cgi?id=944594#c5.

Comment 25

[cyke64]

I can't reproduce this issue : After pressing the recording button, the icon of camera switch is disappeared

B2G: 1.4.0.0-prerelease
Device: Flatfish tablet Tablet Contributor Program

Comment 26

[Andrew Truong [:feer56] [:andrewtruong] [:atruong]]

Is this bug related to the Tablet Contribution Program? The current program have devices running 1.4 and apparently to cyke64 in comment 25, the issue no longer exists. 

This is still UNCO however.

Comment 27

[William Liang]

The device info shows B2G-2.0.0.0-prerelease on the Flatfish devices which has been updated. This bug (932204) does not exist in the current version. However, other problems, such as crash on camera recording & video playback and blank on camera recording preview screen, have occurred instead, which may need to filed as new bugs or recorded in existing bugs.

Comment 28

[Evelyn Hung]

(In reply to cyke64 from comment #25)
> I can't reproduce this issue : After pressing the recording button, the icon
> of camera switch is disappeared
> 
> B2G: 1.4.0.0-prerelease
> Device: Flatfish tablet Tablet Contributor Program

Hi, could you try it on latest master? Thanks!

Comment 29

[Evelyn Hung]

redirect my ni to Sherman for a try.

Comment 30

[Sherman Chen [:chens] (inactive)]

Verified on master, not able to reproduce, camera switch is still on app.

Gaia      0a2c504ae8c6d37e21d04a3b832aba4928f3028d
Gecko     46c9b0371c8c4591e0e8bd5e889a8555e5e1e4ce
BuildID   20140731110216
Version   34.0a1
ro.build.version.incremental=20140725
ro.build.date=Fri Jul 25 01:46:09 CST 2014