939348 - [b2g][Settings] Gralloc alloc takes 40ms, causes Settings page transition to be janky

Status: RESOLVED WORKSFORME

(Core :: Graphics, defect)

Description

[Benoit Girard (:BenWa)]

Profile:
http://people.mozilla.org/~bgirard/cleopatra/#report=c0d4369e8bcde03937538e8e966127ac23797060&select=1223,1967

Comment 1

[Andreas Gal :gal]

Attachment: Part 1: Allocate gralloc buffers on the image bridge thread and re-wrap them into a new actor when sending them to the compositor.

Comment 2

[Andreas Gal :gal]

Attachment: Part 2: Always register the memory reporter from the main thread.

Comment 3

[Sotaro Ikeda [:sotaro]]

I applied the patches to master hamachi. I saw a lot of genlock failures like the following.

01-22 17:42:10.219   137   268 W Adreno200-EGLSUB: <LockImage:1942>: genlock_lock_buffer GENLOCK_READ_LOCK failed
01-22 17:42:10.219   137   268 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_DREADLOCK failed (lockType0x2, err=Invalid argument fd=211)
01-22 17:42:10.219   137   268 W Adreno200-EGLSUB: <LockImage:1942>: genlock_lock_buffer GENLOCK_READ_LOCK failed
01-22 17:42:10.229   137   268 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_DREADLOCK failed (lockType0x2, err=Not a typewriter fd=894)
01-22 17:42:10.229   137   268 W Adreno200-EGLSUB: <LockImage:1942>: genlock_lock_buffer GENLOCK_READ_LOCK failed
01-22 17:42:10.229   137   268 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_DREADLOCK failed (lockType0x2, err=Not a typewriter fd=894)
01-22 17:42:10.229   137   268 W Adreno200-EGLSUB: <LockImage:1942>: genlock_lock_buffer GENLOCK_READ_LOCK failed
01-22 17:42:10.229   137   268 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_DREADLOCK failed (lockType0x2, err=Not a typewriter fd=894)
01-22 17:42:10.229   137   268 W Adreno200-EGLSUB: <LockImage:1942>: genlock_lock_buffer GENLOCK_READ_LOCK failed
01-22 17:42:10.229   137   268 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_DREADLOCK failed (lockType0x2, err=Not a typewriter fd=894)
01-22 17:42:10.229   137   268 W Adreno200-EGLSUB: <LockImage:1942>: genlock_lock_buffer GENLOCK_READ_LOCK failed

Comment 4

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #3)
> I applied the patches to master hamachi. I saw a lot of genlock failures
> like the following.

Some part of ui was rendered correctly. But some part of UI's rendering was not correct. It became noticeable when I changed the UI quickly like APZC.

Comment 5

[Sotaro Ikeda [:sotaro]]

On nexus-4 rendering result seems correct. But when I change the UI quickly like APZC part, the system was sometimes crashed.

Comment 6

[Andreas Gal :gal]

I am allocating on a different thread but otherwise nothing should be different. This is quite surprising.

Comment 7

[Andreas Gal :gal]

Can you catch the crash?

Comment 8

[Andreas Gal :gal]

I think this is an OOM crash.

Comment 11

[Sotaro Ikeda [:sotaro]]

(In reply to Andreas Gal :gal from comment #8)
> I think this is an OOM crash.

Yeah, it was oom crash. But the memory leak seems to happens. After the crash, all applications are kill soon after the application start. It continues until b2g reboot.

I confirmed the above by the following STR.
[1] Open browser app
[2] Move to http://www.yahoo.co.jp/
[3] Do pan and zoom very quickly continuously until browser app's crash.

After the crash, b2g becomes very unstable. No application can exit. When home app is restarted, it is killed because of oom. I saw a a log of the following log.

> E OomLogger: [Kill]: send sigkill to 12222 (Homescreen), adj 134, size 8987

Comment 12

[Sotaro Ikeda [:sotaro]]

It might be better to think about ICS after the problem in Comment 11 is fixed.

Comment 13

[Andreas Gal :gal]

Bas pointed to the bug here. We were recursing ourselves to death.

Comment 14

[Andreas Gal :gal]

Attachment: Part 3: Fix endless recursion.

Comment 15

[Sotaro Ikeda [:sotaro]]

(In reply to Andreas Gal :gal from comment #14)
> Created attachment 8364372 [details] [diff] [review]
> Part 3: Fix endless recursion.

I applied the patch to nexus-4, but the memory leaking and oom after the creash was same. The symptom continued until b2g reboot.

Comment 16

[Andreas Gal :gal]

I don't see the leak. Can you try to track it down? Must be a driver difference.

Comment 17

[Sotaro Ikeda [:sotaro]]

This week, I do not have a time to look into it.

Comment 18

[Andreas Gal :gal]

Ok, I will try to debug it on my end.

Comment 19

[Andreas Gal :gal]

Attachment: Part 1: allocate gralloc buffers on the image bridge only

Comment 20

[Andreas Gal :gal]

Attachment: Part 2: move image bridge onto its own thread

Comment 21

[Nicolas Silva [:nical]]

Comment on attachment 8364973 [details] [diff] [review]
Part 2: move image bridge onto its own thread

Review of attachment 8364973 [details] [diff] [review]:
-----------------------------------------------------------------

Beware, ImageBridgeParent was designed to be in the compositor thread because it does transactions that need to not race with compositing and the transactions that originate from the main thread. If you want to make ImageBridgeParent live in its own thread you need to forward its transactions to the compositor thread in a thread-safe way, and that could be tricky if the transactions are synchronous. If the goal of this patch is to allocate gralloc buffers without waiting for the compositor thread's event loop, I very strongly suggest that this gets done in a an other protocol with its own channel.

Comment 22

[Andreas Gal :gal]

Thanks for the drive-by nical! I will add a separate protocol.

Comment 23

[Andreas Gal :gal]

Actually this is a bit beyond my pay grade. I will need some help wiring up the separate protocol. Ideally each client that spins up a parent actor should get its own thread so content processes don't block each other when they allocate.

Comment 24

[Chiajung Hung [:chiajung]]

I test this patch on Unagi and I observed following error when launch camera

01-27 10:31:44.997  6746  6774 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_LOCK failed (lockType0x2, err=Not a typewriter fd=153)
01-27 10:31:44.997  6746  6774 E Adreno200-EGLSUB: LockImage() genlock_lock_buffer GENLOCK_READ_LOCK failed
01-27 10:31:45.017  6746  6894 E Adreno200-EGL: <qeglDrvAPI_eglMakeCurrent:2693>: EGL_BAD_ACCESS
01-27 10:31:45.017  6746  6894 E libEGL  : eglMakeCurrent:678 error 3002 (EGL_BAD_ACCESS)
01-27 10:31:45.017  6746  6894 E Adreno200-EGL: <qeglDrvAPI_eglMakeCurrent:2693>: EGL_BAD_ACCESS
01-27 10:31:45.017  6746  6894 E libEGL  : eglMakeCurrent:678 error 3002 (EGL_BAD_ACCESS)
01-27 10:31:45.017  6746  6894 E Adreno200-EGL: <qeglDrvAPI_eglMakeCurrent:2693>: EGL_BAD_ACCESS
01-27 10:31:45.017  6746  6894 E libEGL  : eglMakeCurrent:678 error 3002 (EGL_BAD_ACCESS)

and following in Browser app

01-27 10:33:57.731  6924  7098 W memalloc: Falling back to ashmem
01-27 10:33:57.751  6924  6952 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_LOCK failed (lockType0x2, err=Invalid argument fd=559)
01-27 10:33:57.751  6924  6952 E Adreno200-EGLSUB: LockImage() genlock_lock_buffer GENLOCK_READ_LOCK failed
01-27 10:33:57.751  6924  7098 D memalloc: ashmem: Allocated buffer base:0x4ba00000 size:2027520 fd:76
01-27 10:33:57.761  7097  7123 D memalloc: ashmem: Mapped buffer base:0x45400000 size:2027520 fd:38
01-27 10:33:57.761  6924  6952 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_LOCK failed (lockType0x2, err=Invalid argument fd=559)
01-27 10:33:57.761  6924  6952 E Adreno200-EGLSUB: LockImage() genlock_lock_buffer GENLOCK_READ_LOCK failed
01-27 10:33:57.761  6924  7098 D memalloc:  Out of PMEM. Dumping PMEM stats for debugging
01-27 10:33:57.761  6924  7098 D memalloc:  ------------- PRINT PMEM STATS --------------
01-27 10:33:57.761  6924  7098 D memalloc:  Node 0 -> Start Address : 0 Size 19200 Free info 0
01-27 10:33:57.761  6924  7098 D memalloc:  Node 1 -> Start Address : 19200 Size 19200 Free info 0
01-27 10:33:57.761  6924  7098 D memalloc:  Node 2 -> Start Address : 38400 Size 51840 Free info 1
01-27 10:33:57.761  6924  7098 D memalloc:  Node 3 -> Start Address : 90240 Size 19200 Free info 0
01-27 10:33:57.761  6924  7098 D memalloc:  Node 4 -> Start Address : 109440 Size 33920 Free info 1
01-27 10:33:57.761  6924  7098 D memalloc:  Node 5 -> Start Address : 143360 Size 19200 Free info 0
01-27 10:33:57.761  6924  7098 D memalloc:  Node 6 -> Start Address : 162560 Size 67584 Free info 0
01-27 10:33:57.771  6924  7098 D memalloc:  Node 7 -> Start Address : 230144 Size 32000 Free info 1
01-27 10:33:57.771  6924  7098 D memalloc:  Total Allocated: Total Free: 
01-27 10:33:57.771  6924  7098 D memalloc: ----------------------------------------------
01-27 10:33:57.771  6924  7098 E memalloc: /dev/pmem: No more pmem available
01-27 10:33:57.771  6924  7098 W memalloc: Falling back to ashmem
01-27 10:33:57.801  6924  6952 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_LOCK failed (lockType0x2, err=Invalid argument fd=559)
01-27 10:33:57.801  6924  6952 E Adreno200-EGLSUB: LockImage() genlock_lock_buffer GENLOCK_READ_LOCK failed
01-27 10:33:57.801  6924  7098 D memalloc: ashmem: Allocated buffer base:0x4c022000 size:2027520 fd:557
01-27 10:33:57.811  6924  6952 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_LOCK failed (lockType0x2, err=Invalid argument fd=559)
01-27 10:33:57.811  6924  6952 E Adreno200-EGLSUB: LockImage() genlock_lock_buffer GENLOCK_READ_LOCK failed
01-27 10:33:57.831  6924  6935 I Gecko   : [Parent 6924] WARNING: pipe error (281): Connection reset by peer: file /home/chiajung/FirefoxOS/unagi/B2G/gecko/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 446
01-27 10:33:57.841  6924  7098 D memalloc: /dev/pmem: Freeing buffer base:0x4a4f6000 size:2162688 offset:5201920 fd:107
01-27 10:33:57.841  6924  6952 E libgenlock: perform_lock_unlock_operation: GENLOCK_IOC_LOCK failed (lockType0x0, err=Invalid argument fd=559)
01-27 10:33:57.841  6924  6952 E Adreno200-EGLSUB: UnlockImage() genlock_unlock_buffer failed

Comment 25

[Chiajung Hung [:chiajung]]

Comment on attachment 8364972 [details] [diff] [review]
Part 1: allocate gralloc buffers on the image bridge only

Review of attachment 8364972 [details] [diff] [review]:
-----------------------------------------------------------------

There is crash after apply the patch,

Comment 26

[Chiajung Hung [:chiajung]]

Comment on attachment 8364973 [details] [diff] [review]
Part 2: move image bridge onto its own thread

Review of attachment 8364973 [details] [diff] [review]:
-----------------------------------------------------------------

This will cause Camera crash whole system

Comment 27

[Benoit Girard (:BenWa)]

We've made several improvements to gralloc allocation performance since this bug was filed. This is probably no longer applicable to the settings app.