1529629 - Use overlay composition for fullscreen video on Android

Status: NEW

(Core :: Graphics, enhancement, P3)

Description

[Jeff Muizelaar [:jrmuizel]]

My understanding is that Chrome does this.
Here's the issue that added it to Chrome:
https://bugs.chromium.org/p/chromium/issues/detail?id=618368&desc=2

Comment 1

[Sotaro Ikeda [:sotaro]]

If we want to do it with compositor on android, it might be easier to implement it like obsoleted b2g hw composer.

Comment 2

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #1)

If we want to do it with compositor on android, it might be easier to implement it like obsoleted b2g hw composer.

I expected to use SurfaceView as a overlay. But ANativeWindow does not expose a API for just setting GraphicBuffer(AHardwareBuffer) for rendering. Then it needs extra EGL copy.

From :jhlin's info. chromium already did similar thing.
https://bugs.chromium.org/p/chromium/issues/detail?id=618368&desc=2

It created a dialog and took its Surface for overlay. It also does extra EGL copy to the Surface. But it improves performance on chromium. Since, on chromium, content is rendered by compositor for content, then the result is rendered by compositor for chrome. But on latest chromium, ImageReaderGLOwner seems to be preferred than SurfaceTextureGLOwner if possible.
https://cs.chromium.org/chromium/src/media/gpu/android/video_frame_factory_impl.cc?l=58
https://cs.chromium.org/chromium/src/media/gpu/android/texture_owner.cc?l=49

ImageReaderGLOwner uses AImageReader, AImage and AHardwareBuffer. AHardwareBuffer usage has less restrictions than using SurfaceTexture directly. But AImageReader usage is disabled on several gpus.
https://cs.chromium.org/chromium/src/gpu/config/gpu_driver_bug_list.json?l=2932
https://cs.chromium.org/chromium/src/gpu/config/gpu_driver_bug_list.json?l=3014

When I looked into latest chromium source code, I found GLSurfaceEGLSurfaceControl. It uses SurfaceControl via ASurfaceControl and ASurfaceTransaction.
https://cs.chromium.org/chromium/src/ui/gl/gl_surface_egl_surface_control.cc
https://cs.chromium.org/chromium/src/ui/gl/android/android_surface_control_compat.h
https://cs.chromium.org/chromium/src/ui/gl/android/android_surface_control_compat.cc

ASurfaceTransaction_setBuffer() could set ASurfaceControl and AHardwareBuffer. It seems possible to set a buffer to overlay directly without copy.
https://cs.chromium.org/chromium/src/ui/gl/gl_surface_egl_surface_control.cc?l=200
https://cs.chromium.org/chromium/src/ui/gl/gl_surface_egl_surface_control.cc?l=301

But ASurfaceTransaction does not exist even in Android Pie. It seems to be added in Android Q. I found it in surface_control.h in android ndk r20 beta 3. As a preparation of ASurfaceTransaction, it might better to add AHardwareBuffer support.
https://developer.android.com/preview/features#surface
https://developer.android.com/ndk/downloads#beta-downloads