1008000 - [Flame] When recording a video in the camera app, the viewfinder is not smooth when moving the phone

Status: RESOLVED FIXED

(Firefox OS Graveyard :: GonkIntegration, defect, P1)

Description

[Jason Smith [:jsmith]]

Build

* Date: 5/8/2014
* Version: 1.4
* Device: Flame
* Gaia: 4ce973ef0732b0d52cb043210db598aa176b2ce9
* Gecko: 16ab7f6b18f8
* Build ID: 20140508000201

STR

1. Open camera
2. Start a recording
3. Start slowly moving the phone

Expected

The camera viewfinder should smoothly transition as the phone moves.

Actual

The camera viewfinder is jumpy & not smooth. Interestingly enough, the recording comes out fine, even though viewfinder made the camera recording look jumpy.

Comment 1

[Jason Smith [:jsmith]]

https://www.youtube.com/watch?v=W6eT5KtqbOo&feature=youtu.be

Comment 2

[Jason Smith [:jsmith]]

Works fine on Open C btw.

Comment 3

[Jason Smith [:jsmith]]

Attachment: Logcat

Comment 4

[Jason Smith [:jsmith]]

Francis - Can you outreach to the partner for Flame on this?

Comment 5

[Francis Lee [:frlee]]

Flame is shipped with v1.3. this is agree in the contract. v1.3 works well and there's no viewfinder slow moving problem. i believe before we request vendor to investigate, we need to make sure it's not FxOS related issue.

although Flame and Open C both use 8210 chipset, but RIL is different. Flame has dsds support. RIL level is definitely different from Open C. it may not be convincible to claim it's vendor's issue since Open C works with our 1.4 gaia/gecko.

QCT won't release 1.4 QCT RIL based on 8210 chipset until E/June. (QCT's current targeted schedule). it will be better to test with 1.4 QCT RIL, or we should test with our own Moz' RIL to ensure RIL level is same for both Open C and Flame.

Comment 6

[Jason Smith [:jsmith]]

Mike - Can you weigh in here if you agree or disagree this is a vendor problem?

Comment 7

[Jason Smith [:jsmith]]

Also - this was tested with Flame in a 512 MB situation.

Comment 8

[Mike Lien[:mlien]]

I noticed that b2g process always consumes high CPU time even it's idle
if memory set to 512MB, CPU time from 60~70%
if memory set to 1GB, CPU time from 50~60%

I think this is why performance will be much worse

Comment 9

[Mike Habicher [:mikeh] (high bugzilla latency)]

High CPU consumption will definitely be a problem.

Can you confirm if the laggy viewfinder problem goes away if you switch to video mode? if you switch -back- to camera mode?

Comment 10

[Jason Smith [:jsmith]]

Adding qawanted to check comment 9 & find out if this occurs on the 1.3 Flame base image.

Comment 11

[Mike Habicher [:mikeh] (high bugzilla latency)]

Sotaro, I'm out of the office next week. Can you please look into this issue?

Comment 12

[Sotaro Ikeda [:sotaro]]

Okey, I can take a look. Bug 988704 might be related to this bug.

Comment 13

[Sotaro Ikeda [:sotaro]]

assign to me until mikeh comes back.

Comment 14

[Chris Kreinbring [:CKreinbring]]

Did not repro on the Flame 1.3 base image.  The video remained smooth when the camera was moved while recording.

Switching back to camera mode and returning to video mode does mot remove the lag from recording video.

Comment 15

[Chris Kreinbring [:CKreinbring]]

Repros on the earliest available Flame 1.4 build.  It's noticeable, but not as blatant as today's build.

BuildID: 20140421160203
Gaia: 26432916d10434103a822f17be4624a342cadfba
Gecko: 211431128e5b
Version: 30.0a2
Firmware Version: V10F-3

Comment 16

[Sotaro Ikeda [:sotaro]]

Before to start investigating, I faced the Bug 1009297. When flame device is flashed by ".flash.sh", video decoding and camera recording does not work. I already have a fix.

Comment 17

[Sotaro Ikeda [:sotaro]]

Actually in video decoding case, omx_vdec::allocate_input_buffer() was failed because of lacking USE_ION definition. Actual place was the following.

>         pmem_fd = open (MEM_DEVICE,O_RDWR);

https://www.codeaurora.org/cgit/quic/la/platform/hardware/qcom/media/tree/mm-video-v4l2/vidc/vdec/src/omx_vdec_msm8974.cpp?h=jb_3.2_rb5.56#n4668

Comment 18

[Sotaro Ikeda [:sotaro]]

Sorry, the above comment 17 is for Bug 1009297.

Comment 19

[Wilson Page [:wilsonpage]]

Seems pretty smooth on latest 'master' build for me. Nothing out of the ordinary.

Comment 20

[Sotaro Ikeda [:sotaro]]

> 
> The camera viewfinder is jumpy & not smooth. Interestingly enough, the
> recording comes out fine, even though viewfinder made the camera recording
> look jumpy.

In mozCamera case, recording stream and preview stream are totally different stream. Therefore they are basically independent each other.

Comment 21

[Mike Lien[:mlien]]

I find out high CPU time is due to ril problem
Before I enable moz-ril (v10F-3 base image + v1.4 gaia/gecko), CPU time is as comment 8 mentioned
You can also use adb shell top -m 10 -t to get more detailed information
  PID   TID PR CPU% S     VSS     RSS PCY UID      Thread          Proc
  308   484  0  43% R 168112K  71464K     root     Gecko_IOThread  /system/b2g/b2g
  308   308  0  32% S 168112K  71464K     root     b2g             /system/b2g/b2g
  918   918  1   1% R   1268K    600K     root     top             top

Thread Gecko_IOThread and b2g consume lot of CPU time

After I enable moz-ril, CPU time is back to normal (near to 0% while idle) and camera not smooth problem is gone

Comment 22

[Sotaro Ikeda [:sotaro]]

From Comment 21, it seems gonk integration problem. It is not related to camera.

Comment 23

[Sotaro Ikeda [:sotaro]]

Based on comment 21, I un-assign from the bug.

Comment 24

[Sotaro Ikeda [:sotaro]]

I locally applied nfc bug (Bug 1001327). It seems to fix the camera preview and cpu occupied problem.

Comment 25

[Sotaro Ikeda [:sotaro]]

I checked camera preview in the following combinations.

- [1] Build v1.4 flame ROM from source
      + camera preview is slow
- [2] Build master flame ROM from source
      + camera preview is quick
- [3] Build v1.4 flame ROM from source + master gecko+gaia from [2]
      + camera preview is slow
- [4] Build master flame ROM from source + v1.4 gecko+gaia from [1]
      + camera preview is quick

Comment 26

[Sotaro Ikeda [:sotaro]]

> 
> Thread Gecko_IOThread and b2g consume lot of CPU time
> 
> After I enable moz-ril, CPU time is back to normal (near to 0% while idle)
> and camera not smooth problem is gone

When I built v1.4 flame from source, moz-ril seems to be used in the ROM. But camera preview is still slow. From Comment 24, this case seems to be affected by nfc.

Comment 27

[Sotaro Ikeda [:sotaro]]

In built v1.4 flame from source case, cpu usage in camera preview was the follwoing. It is similar to Comment 21.

User 20%, System 45%, IOW 0%, IRQ 0%
User 126 + Nice 2 + Sys 280 + Idle 200 + IOW 6 + IRQ 0 + SIRQ 0 = 614

  PID   TID PR CPU% S     VSS     RSS PCY UID      Thread          Proc
  327   501  1  30% R 182640K  75400K     root     Gecko_IOThread  /system/b2g/b2g
  327   327  1  25% S 182640K  75400K     root     b2g             /system/b2g/b2g
  373  1182  0   1% S  38920K  10040K     camera   mm-qcamera-daem /system/bin/mm-qcamera-daemon
  327   700  0   1% S 182640K  75400K     root     Compositor      /system/b2g/b2g

Comment 28

[Kai-Chih Hu [:khu]]

I tried 2.0 build on Flame and it works fine.

Comment 29

[Kai-Chih Hu [:khu]]

Wesley / Ken, although it sounds weird, can we get your comments on this bug? Thanks.

Comment 30

[Kai-Chih Hu [:khu]]

(In reply to Sotaro Ikeda [:sotaro] from comment #26)
> When I built v1.4 flame from source, moz-ril seems to be used in the ROM.
> But camera preview is still slow. From Comment 24, this case seems to be
> affected by nfc.

Oops. I forgot to quote what Sotaro said here. Ken, Wesley, FYI.

Comment 31

[Vincent Liu[:vliu]]

I got the below gaia message when I start camera recording.

"Video not recorded. An error prevented Camera from recording the video."

I can't do any further test for this bug without the fix I met. I will quickly figure it out and than go back to this bug. Or anyone can comment for me if you have any idea for what I meet currently. Thanks.

Comment 32

[Vincent Liu[:vliu]]

Attachment: log-for-bug-1008000.log

I tried to fix the base image to 10F to look into this issue, the issue was able to reproduce easily.

Base image: 10F
Gecko (v1.4) : c1b9b2a8cf2a852384f9ae408117303fb35932ef
Gaia (v1.4) : 233dd43b3b976e66a619dbc1b4044ed1e3ca3e34

When the issue happened, the log kept printing the below messages.

05-22 03:05:00.390   291   480 I I/O     : connect failed with error 2 (No such file or directory)
05-22 03:05:00.390   291   480 I I/O     : connect failed with error 2 (No such file or directory)
05-22 03:05:00.390   291   480 I I/O     : connect failed with error 2 (No such file or directory)
05-22 03:05:00.390   291   480 I I/O     : connect failed with error 2 (No such file or directory)
05-22 03:05:00.390   291   480 I I/O     : connect failed with error 2 (No such file or directory)
05-22 03:05:00.390   291   480 I I/O     : connect failed with error 2 (No such file or directory)

The detail log message was also attached.

Comment 33

[Vincent Liu[:vliu]]

Here to explain more that the device start printing (I I/O     : connect failed....) during the boot. No matter what app I operated, it always keep printing. Strictly speaking, the whole performance of this device became sluggish during the operating.

From gdb tracking, I found it connected fail in UnixFdWatcher.cpp::OnError(). @Thomas, it seems you are familiar with this part. Would you please put out any idea to move next? Many thanks. 

#0  mozilla::ipc::UnixFdWatcher::OnError (this=0xb036b2c0, aFunction=0xb5f628d4 "connect", aErrno=2) at ../../../gecko/ipc/unixfd/UnixFdWatcher.cpp:93
#1  0xb4f4e26c in mozilla::ipc::UnixSocketImpl::OnError (this=0xb036b2c0, aFunction=<optimized out>, aErrno=<optimized out>) at ../../../gecko/ipc/unixsocket/UnixSocket.cpp:638
#2  0xb4f4dacc in mozilla::ipc::UnixSocketWatcher::Connect (this=0xb036b2c0, aAddr=<optimized out>, aAddrLen=<optimized out>) at ../../../gecko/ipc/unixfd/UnixSocketWatcher.cpp:38
#3  0xb4ea4d90 in MessageLoop::RunTask (this=0xb3fcfde0, task=0xb015cc30) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:344
#4  0xb4ea5452 in MessageLoop::DeferOrRunPendingTask (this=<optimized out>, pending_task=<optimized out>) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:352
#5  0xb4ea6484 in DoWork (this=<optimized out>) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:430
#6  MessageLoop::DoWork (this=0xb3fcfde0) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:409
#7  0xb4e9ddbe in base::MessagePumpLibevent::Run (this=0xb6b01ee0, delegate=0xb3fcfde0) at ../../../gecko/ipc/chromium/src/base/message_pump_libevent.cc:311
#8  0xb4ea4d1e in MessageLoop::RunInternal (this=<optimized out>) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:226
#9  0xb4ea4dd0 in RunHandler (this=0xb3fcfde0) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:219
#10 MessageLoop::Run (this=0xb3fcfde0) at ../../../gecko/ipc/chromium/src/base/message_loop.cc:193
#11 0xb4ea75c2 in base::Thread::ThreadMain (this=0xb6b1a500) at ../../../gecko/ipc/chromium/src/base/thread.cc:162
#12 0xb4e9e270 in ThreadFunc (closure=<optimized out>) at ../../../gecko/ipc/chromium/src/base/platform_thread_posix.cc:39
#13 0xb6f1fba4 in __thread_entry (func=0xb4e9e269 <ThreadFunc(void*)>, arg=0xb6b1a500, tls=0xb3fcff00) at bionic/libc/bionic/pthread_create.cpp:92
#14 0xb6f1fd20 in pthread_create (thread_out=0xb6b1a508, attr=<optimized out>, start_routine=0x78, arg=0xb6b1a500) at bionic/libc/bionic/pthread_create.cpp:201

Comment 34

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Hi,

I don't have a Flame for further debugging, so I'm just guessing here.

The error message indicates that you're trying to connect to a Unix Local socket (AF_LOCAL or AF_UNIX) that doesn't exists. So you should try to instrument mozilla::ipc::UnixSocketWatcher::Connect() to see, which file path is about to be opened. See [1] for a description of struct sockaddr.

I'm surprised to see this problem in the context of Video recordings. We only use this code for IPC with system daemons.

[1] https://www.gnu.org/software/libc/manual/html_node/Address-Formats.html#Address-Formats

Comment 35

[Vincent Liu[:vliu]]

(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #34)
> Hi,
> 
> I don't have a Flame for further debugging, so I'm just guessing here.
> 
> The error message indicates that you're trying to connect to a Unix Local
> socket (AF_LOCAL or AF_UNIX) that doesn't exists. So you should try to
> instrument mozilla::ipc::UnixSocketWatcher::Connect() to see, which file
> path is about to be opened. See [1] for a description of struct sockaddr.
> 

Thanks for the suggestion.

> I'm surprised to see this problem in the context of Video recordings. We
> only use this code for IPC with system daemons.
> 
> [1]
> https://www.gnu.org/software/libc/manual/html_node/Address-Formats.
> html#Address-Formats

The log (I I/O     : connect failed....) has started printing since the device runs in booting. I think there is no any connection with video recording.

Comment 36

[Vincent Liu[:vliu]]

Since the current issue I see is not about video recording and I am not sure if it affects the original problem. I will file another bug and set block against this.

Comment 37

[Sotaro Ikeda [:sotaro]]

jeff, you talked about v1.4 flame animation is slow. This seems root cause of it.

Comment 38

[Vincent Liu[:vliu]]

I confirmed that the root cause of bug 1014485 consumes system performance which also including video recording.

Comment 39

[Ken Chang[:kenkai]]

According to comment 38, we know the root cause.

Comment 40

[Wesley Huang [:wesley_huang] (EPM) (NI me)]

depends on bug 1010993 and bug 1014485

Comment 41

[Sotaro Ikeda [:sotaro]]

:vliu, is this bug is already fixed in v1.4? If it is not fixed yet, can we FIX this bug soon?

Comment 42

[Vincent Liu[:vliu]]

(In reply to Sotaro Ikeda [:sotaro] from comment #41)
> :vliu, is this bug is already fixed in v1.4? If it is not fixed yet, can we
> FIX this bug soon?

Yes, This bug is already fixed since the patch of bug 1014485 has landed.

Comment 43

[Mike Lee [:mlee]]

Resolving as FIXED per comment 42.