1562818 - [meta] Add support of AHardwareBuffer on Android

Status: NEW

(Core :: Graphics: WebRender, enhancement, P3)

Description

[Sotaro Ikeda [:sotaro]]

AHardwareBuffer is defined by ndk. It is a wrapper of android::GraphicBuffer. It was introduced in Android 8.0 (API 26). AHardwareBuffer is more flexible than android::SurfaceTexture, since SurfaceTexture has several limitations.

https://developer.android.com/ndk/guides/stable_apis#hardware_buffer_apis
https://developer.android.com/ndk/reference/group/a-hardware-buffer
https://developer.android.com/reference/android/hardware/HardwareBuffer.html

Comment 1

[Sotaro Ikeda [:sotaro]]

https://searchfox.org/mozilla-central/source/mobile/android/confvars.sh#24

Comment 2

[Sotaro Ikeda [:sotaro]]

The following helps to know how to use AHardwareBuffer.
https://medium.com/@spencerfricke/android-ahardwarebuffer-shared-memory-over-unix-domain-sockets-7b27b1271b36
https://github.com/sjfricke/NDK-Socket-IPC

Comment 3

[Kris Taeleman (:ktaeleman)]

Hey Sotaro,

What would be the use cases and what benefits do you see us getting from this?

Comment 4

[Sotaro Ikeda [:sotaro]]

What would be the use cases and what benefits do you see us getting from this?

SurfaceTexture/Surface has several limitations.

• [1] Consumer could exist only in one process and one thread.
  • If content side want to get access to the data, it needs to copy back from compositor to client side. It is implemented for video rendering to WebGL by Bug 1486659. But it does not work for all use cases like Bug 1526207 and offscreencanvas.
  • https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_SurfaceAllocatorService_68.pdf
• [2] Consumer provide only one buffer access that exists on top.
  • During video rendering SurfaceTexture uses many buffers, only one top frame could be accessed via SurfaceTexture on consumer side.
  • https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_CodecProxy_68.pdf
• [3] SurfaceTexture does not provide access to raw buffer
  • It is necessary to implement GetAsSourceSurface(). See Bug 1526207

AHardwareBuffer(android::GraphicBuffer) could remove the above limitations. Then, content process could access to a buffer without copy-back from compositor. And could implement GetAsSourceSurface() on any thread in content process. Further AHardwareBuffer could be used to implement video/WebGL overlay with ASurfaceTransaction that exists since android10(Q). See Bug 1529629.

Comment 5

[Darkspirit]

(Sotaro Ikeda [:sotaro] from bug 1529629 comment 1)

it might be easier to implement it like obsoleted b2g hw composer.

I found bug 787048 and other gralloc bugs: https://bugzilla.mozilla.org/buglist.cgi?quicksearch=gralloc&list_id=15282078

https://source.android.com/devices/neural-networks/ahardwarebuffer

AHardwareBuffer is a type of shared memory that wraps a gralloc buffer.

https://software.intel.com/content/www/us/en/develop/articles/native-one-copy-texture-uploads-for-chrome-os-on-intel-architecture-enabled-by-default.html

A while ago Chromium introduced GpuMemoryBuffer to eliminate CPU to GPU copies. On the implementation side, GpuMemoryBuffer is a Linux DMA buffer on Chrome OS, IOSurface on Mac OS X, and a gralloc buffer on Android. The software fallback uses POSIX shared memory.

Mac: bug 1403618
Linux: bug 1629788
Therefore I assume bug 1617051 is more a duplicate of this bug.