[flatfish][camera] After pressing the recording button, the icon of camera switch is disappeared

RESOLVED WORKSFORME

Status

Firefox OS
Gaia::Camera
RESOLVED WORKSFORME
4 years ago
4 years ago

People

(Reporter: JustinLee, Assigned: vliu)

Tracking

(Blocks: 1 bug)

unspecified
1.3 Sprint 6 - 12/6
ARM
Gonk (Firefox OS)
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [flatfish][TCP][POVB])

Attachments

(1 attachment, 1 obsolete attachment)

1.08 KB, patch
Details | Diff | Splinter Review
(Reporter)

Description

4 years ago
Steps to reproduce:
1. Open Camera app
2. Switch to video recording mode
3. Press "video recording" button => pop-up a message "Video not recorded. An error..... the video."
4. Press "OK"

Actual results:
The icon of camera switch is disappeared. The user can't switch front/rear camera.
(Reporter)

Updated

4 years ago
Blocks: 903304
(Assignee)

Comment 1

4 years ago
Currently narrow down the fail point in 

http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkRecorder.cpp#1269

I still figure out why it fails when it called.
(Assignee)

Comment 2

4 years ago
OMXCodec::Create() failed because encoder_flags always be 0. Backed trace the code and currently narrow down the fail point in  

http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkCameraSource.cpp#482

Updated

4 years ago
blocking-b2g: --- → koi?
Whiteboard: [Flatfish only]

Updated

4 years ago
Blocks: 932672
Whiteboard: [Flatfish only] → [Flatfish only][developer+]

Comment 3

4 years ago
This is due to Video Recording patches are not integrated in the Main-stream line.
Could you integrate patches(4) from the below link and test it.
https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c31

Initailly I faced same, after this patch for Video recording integration, start working for me.

Comment 4

4 years ago
Hi! Vincent,

Please give it a try. Thanks.

--
Kwvwn
Assignee: nobody → vliu
(Assignee)

Comment 5

4 years ago
(In reply to saminath from comment #3)
> This is due to Video Recording patches are not integrated in the Main-stream
> line.
> Could you integrate patches(4) from the below link and test it.
> https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c31
> 
> Initailly I faced same, after this patch for Video recording integration,
> start working for me.

I tried you suggestion but still fail. May I confirm your steps.

1. Which device you used in your test?
2. In the patch you showed to me, it only change b2g.sh to use a umask of 0007 instead of 0027, right?

Thanks.

Comment 6

4 years ago
Hi Vincent.

1. Which device you used in your test?
I tested on Nexus-4 device

2. In the patch you showed to me, it only change b2g.sh to use a umask of 0007 instead of 0027, right?

No. Please do integration all 4 Patches in the attachment (0001-HACK-Change-umask-to-007.patch,0001-HACK-Mount-internal-sdcard-to-data-sdcard-and-change.patch,bug-910498-getVol-n4.patch,bug-910498-hack.patch).

Thanks.

Comment 7

4 years ago
Taipei triage result: ---.
blocking-b2g: koi? → ---

Comment 8

4 years ago
This is functional fail of Flatfish camera.  Hi, Francis: please confirm.
Flags: needinfo?(frlee)

Comment 9

4 years ago
for comment #7, the reason is: until we have final decision of Flatfish base version (v1.2 or v1.3), change to --- for now.

for comment #8, Raymond, this is an issue we must fix.
Flags: needinfo?(frlee)
(Assignee)

Comment 10

4 years ago
(In reply to saminath from comment #6)
> Hi Vincent.
> 
> 1. Which device you used in your test?
> I tested on Nexus-4 device
> 
> 2. In the patch you showed to me, it only change b2g.sh to use a umask of
> 0007 instead of 0027, right?
> 
> No. Please do integration all 4 Patches in the attachment
> (0001-HACK-Change-umask-to-007.patch,0001-HACK-Mount-internal-sdcard-to-data-
> sdcard-and-change.patch,bug-910498-getVol-n4.patch,bug-910498-hack.patch).
> 
> Thanks.

As I observed on Flatfish, it is not the same case I saw on Nexus-4. Nexus-4 get fails on the blow link.

http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkCameraControl.cpp#1087

But for Flatfish, it didn't block on GetVolumeByPath(..). Please see the comment 2 to know the actual block point. Thanks.
as we confirmed that Flatfish will use v1.3 and we need to respect current testing cycle, set target milestone to 12/6 since 1.3FC tag is not created yet.
blocking-b2g: --- → 1.3+
Target Milestone: --- → 1.3 Sprint 6 - 12/6
(Assignee)

Comment 12

4 years ago
(In reply to Vincent Liu[:vliu] from comment #2)
> OMXCodec::Create() failed because encoder_flags always be 0. Backed trace
> the code and currently narrow down the fail point in  
> 
> http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkCameraSource.
> cpp#482

Hi JustinLee,

I traced the code and it failed in the link in Comment 2. |mCameraHw->StoreMetaDataInBuffers(true)| binds to libcameraservice in mediaserver and then calls to CameraHardware. Can you also find this case? I need your help to figure out why it returns fail. Thanks
Flags: needinfo?(hungche)
(Assignee)

Comment 13

4 years ago
Created attachment 830082 [details]
WIP-1.txt

The attached file is a WIP. With this patch, camera recording won't be blocked by a gaia message "video not recorded..." and has a recording status. But when I tried to press the button to stop the recording, it won't store any video file. For this, I still look into it.
(Reporter)

Comment 14

4 years ago
Hi Vincent,

I traced the code and found that the return value of storeMetaDataInBuffers(bool enable) is UNKNOWN_ERROR only in camera HAL on flatfish.
FYR.
Flags: needinfo?(hungche)
(Assignee)

Comment 15

4 years ago
When Camera app start video recording, OMXCodec received a message with |event = OMX_EventError|

https://android.googlesource.com/platform/frameworks/av/+/android-4.2.2_r1.2/media/libstagefright/OMXCodec.cpp

The detailed message showed below

(gdb) p msg
$3 = (const android::omx_message &) @0x428fdd58: {type = android::omx_message::EVENT, node = 0x1, u = {
    event_data = {event = OMX_EventError, data1 = 2147487744, data2 = 0}, buffer_data = {buffer = 0x1}, 
    extended_buffer_data = {buffer = 0x1, range_offset = 2147487744, range_length = 0, flags = 0, timestamp = 
    -5527349839097065472, platform_private = 0x42e77a48, data_ptr = 0x0}}}

I need help from partner to look into why OMX_EventError is received by OMXCodec. Thanks
Flags: needinfo?(hungche)
(Reporter)

Comment 16

4 years ago
OMX_EventError data1 is 214748774 (in Dec., the HEX is 0x80001000), the error means OMX_ErrorInsufficientResources. I think that video format may be wrong.

In addition, camera app in gaia uses 3gpp as default video format (Below shows the partial code of camera.js). The video format of Android camera app is mp4 on flatfish platform.

http://mxr.mozilla.org/gaia/source/apps/camera/js/camera.js#31
var dcfConfig = {
    key: 'dcf_key',
    seq: null,
    postFix: 'MZLLA',
    prefix: {video: 'VID_', image: 'IMG_'},
    ext: {video: '3gp', image: 'jpg'}
  };

http://mxr.mozilla.org/gaia/source/apps/camera/js/camera.js#658
var dummyblob = new Blob([''], {type: 'video/3gpp'});

Video format may be a key point on video recording.


BTW, the video format of video recording on onetouchFire is 3gpp. Dose it use 3gpp software encode?
Flags: needinfo?(hungche) → needinfo?(vliu)
(Assignee)

Comment 17

4 years ago
(In reply to JustinLee from comment #16)
> OMX_EventError data1 is 214748774 (in Dec., the HEX is 0x80001000), the
> error means OMX_ErrorInsufficientResources. I think that video format may be
> wrong.
> 
> In addition, camera app in gaia uses 3gpp as default video format (Below
> shows the partial code of camera.js). The video format of Android camera app
> is mp4 on flatfish platform.
> 
> http://mxr.mozilla.org/gaia/source/apps/camera/js/camera.js#31
> var dcfConfig = {
>     key: 'dcf_key',
>     seq: null,
>     postFix: 'MZLLA',
>     prefix: {video: 'VID_', image: 'IMG_'},
>     ext: {video: '3gp', image: 'jpg'}
>   };
> 
> http://mxr.mozilla.org/gaia/source/apps/camera/js/camera.js#658
> var dummyblob = new Blob([''], {type: 'video/3gpp'});
> 
> Video format may be a key point on video recording.
> 
> 
> BTW, the video format of video recording on onetouchFire is 3gpp. Dose it
> use 3gpp software encode?

Gaia createed a dummyblob by assigning ext and type for a container. The actual output format for video encoding for a container is retrieved from below

http://mxr.mozilla.org/mozilla-central/source/dom/camera/GonkRecorderProfiles.cpp#80

If you dump mPlatformOutputFormat, you will get OUTPUT_FORMAT_MPEG_4.
Flags: needinfo?(vliu)
(Assignee)

Comment 18

4 years ago
Currently the error state happens because Codec's input buffers are too small to accomodate buffer read from source. Please see the below code snippet in frameworks/av/media/libstagefright/OMXCodec.cpp.

OMXCodec::drainInputBuffer(BufferInfo *info) {

        if (srcBuffer->range_length() > remainingBytes) {
            if (offset == 0) {
                ALOGE(
                     " Codec's input buffers are too small to accomodate "
                     "buffer read from source (info->mSize = %d, srcLength = %d)",
                     info->mSize, srcBuffer->range_length());
                
                srcBuffer->release();
                srcBuffer = NULL;

                setState(ERROR);
                return false;
            }
}

I will then focus on how to determine the buffer size allocation for both codec input buffer and source buffer. Please feel free to let me know relative information if you well known about this. Thanks.
(Assignee)

Comment 19

4 years ago
The value of info->mSize is assigned from calling the OMXCodec::setVideoInputFormat(...). Compared with Nexus-4 and the value seems normal. But for srcBuffer->range_length(), the value is much larger then Nexus-4. The value of srcBuffer->range_length() is assigned by calling the GonkCameraSource::dataCallbackTimestamp(...). This callback function is wrapped by CameraHardware in camera.flatfish.so through HAL. 

#0  android::CameraClient::dataCallbackTimestamp (timestamp=<optimized out>, msgType=32, 
    dataPtr=..., user=0x0)
    at frameworks/av/services/camera/libcameraservice/CameraClient.cpp:753
#1  0x404ffbd8 in android::CameraHardwareInterface::__data_cb_timestamp (
    timestamp=<optimized out>, msg_type=32, data=<optimized out>, index=0, user=0x416e6788)
    at frameworks/av/services/camera/libcameraservice/CameraHardwareInterface.h:484
#2  0x40a6f65e in android::CallbackNotifier::onNextFrameSW(void const*) ()
   from /home/vliu-laptop/projs/a31-422/out/target/product/flatfish/system/lib/hw/camera.flatfish.so
#3  0x40a75a68 in android::V4L2CameraDevice::previewThread() ()
   from /home/vliu-laptop/projs/a31-422/out/target/product/flatfish/system/lib/hw/camera.flatfish.so

I think the next move should focus on fr 2.
(Assignee)

Updated

4 years ago
Duplicate of this bug: 942094
(Assignee)

Updated

4 years ago
Depends on: 944567
(Assignee)

Comment 21

4 years ago
Created attachment 8345644 [details] [diff] [review]
WIP.patch

Here to summarize some issues we'd met for this issue.

1. AAC recording was not support on JB 4.2.2 base. Bug 944567 was filed for this issue tracking.
2. mCameraHw->StoreMetaDataInBuffers(true) always returns UNKNOWN_ERROR.
   Action : Partner modified code in camera.flatfish.so to force to return NO_ERROR. They will land to bitbucket server.
3. Codec's Input buffer size was smaller than Source Buffer Size.
   Cause : Camera hardware read 1280X720 as its default resolution. It conflict the the resolution we set from Gecko.
   Action : Modify the resolution from camera.cfg to match to the same with Gecko.
4. Frame rate conflicted for front camera.
   Cause : Gecko read unsupported frame rate and indicated error recording.
   Action : Modified supported setting in camera.cfg
5. SampleDelta in the first stts table was was much larger than expected value.
   Cause 1 : The unit of timestamp Gecko and camera hardware looked different. The unit of timestamp camera hardware was 1000 times than it looked in Gecko.
   Action 1 : A work around modified in GonkCameraSource.cpp. Please see WIP.patch to know the detail.
              Currently it only happens in Flatfish platform. 

   Cause 2 : The first SampleDelta of stts table was much larger than expected value. 
   Action 2 : A work around modified in MPEG4Writer.cpp. Please see WIP.patch to know the detail.
              
   Actually the timestamp from camera hardware is quite larger than we saw from Nexus-4. As my point of view, it should be the root cause for item 5. After discussed with partner, they will keep working on it.
Attachment #830082 - Attachment is obsolete: true
(Assignee)

Updated

4 years ago
Whiteboard: [Flatfish only][developer+] → [Flatfish only][developer+][POVB]
(Reporter)

Comment 22

4 years ago
(In reply to Vincent Liu[:vliu] from comment #21)
> Created attachment 8345644 [details] [diff] [review]
> WIP.patch
> 
> Here to summarize some issues we'd met for this issue.
> 
> 1. AAC recording was not support on JB 4.2.2 base. Bug 944567 was filed for
> this issue tracking.
> 2. mCameraHw->StoreMetaDataInBuffers(true) always returns UNKNOWN_ERROR.
>    Action : Partner modified code in camera.flatfish.so to force to return
> NO_ERROR. They will land to bitbucket server.
> 3. Codec's Input buffer size was smaller than Source Buffer Size.
>    Cause : Camera hardware read 1280X720 as its default resolution. It
> conflict the the resolution we set from Gecko.
>    Action : Modify the resolution from camera.cfg to match to the same with
> Gecko.
> 4. Frame rate conflicted for front camera.
>    Cause : Gecko read unsupported frame rate and indicated error recording.
>    Action : Modified supported setting in camera.cfg
> 5. SampleDelta in the first stts table was was much larger than expected
> value.
>    Cause 1 : The unit of timestamp Gecko and camera hardware looked
> different. The unit of timestamp camera hardware was 1000 times than it
> looked in Gecko.
>    Action 1 : A work around modified in GonkCameraSource.cpp. Please see
> WIP.patch to know the detail.
>               Currently it only happens in Flatfish platform. 
> 
>    Cause 2 : The first SampleDelta of stts table was much larger than
> expected value. 
>    Action 2 : A work around modified in MPEG4Writer.cpp. Please see
> WIP.patch to know the detail.
>               
>    Actually the timestamp from camera hardware is quite larger than we saw
> from Nexus-4. As my point of view, it should be the root cause for item 5.
> After discussed with partner, they will keep working on it.

About item 5, we have modified camera HAL for fixing timestamp issue. 
So far, camera recording don't need WIP.patch (attachment #8345644 [details] [diff] [review]) on flatfish platform.
I'm unsure why this would be 1.3 blocker specifically, as I don't recall work happening for flatfish on any branch other than trunk. Renominating for more discussion.
blocking-b2g: 1.3+ → 1.3?

Comment 25

4 years ago
I can't reproduce this issue : After pressing the recording button, the icon of camera switch is disappeared

B2G: 1.4.0.0-prerelease
Device: Flatfish tablet Tablet Contributor Program
Is this bug related to the Tablet Contribution Program? The current program have devices running 1.4 and apparently to cyke64 in comment 25, the issue no longer exists. 

This is still UNCO however.

Comment 27

4 years ago
The device info shows B2G-2.0.0.0-prerelease on the Flatfish devices which has been updated. This bug (932204) does not exist in the current version. However, other problems, such as crash on camera recording & video playback and blank on camera recording preview screen, have occurred instead, which may need to filed as new bugs or recorded in existing bugs.

Updated

4 years ago
Whiteboard: [Flatfish only][developer+][POVB] → [flatfish][TCP][POVB]

Comment 28

4 years ago
(In reply to cyke64 from comment #25)
> I can't reproduce this issue : After pressing the recording button, the icon
> of camera switch is disappeared
> 
> B2G: 1.4.0.0-prerelease
> Device: Flatfish tablet Tablet Contributor Program

Hi, could you try it on latest master? Thanks!
Flags: needinfo?(cyke64)

Comment 29

4 years ago
redirect my ni to Sherman for a try.
Flags: needinfo?(cyke64) → needinfo?(shchen)
Verified on master, not able to reproduce, camera switch is still on app.

Gaia      0a2c504ae8c6d37e21d04a3b832aba4928f3028d
Gecko     46c9b0371c8c4591e0e8bd5e889a8555e5e1e4ce
BuildID   20140731110216
Version   34.0a1
ro.build.version.incremental=20140725
ro.build.date=Fri Jul 25 01:46:09 CST 2014
Status: UNCONFIRMED → RESOLVED
Last Resolved: 4 years ago
Flags: needinfo?(shchen)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.