Some comments and questions about using GPU process on Android. - [1] GeckoProcessManager needs to create GPU process + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_GeckoProcessManager_68.pdf - [2] Move SurfaceAllocatorService to GPU process + GonkSurface is used for WebGL and video decoding by MediaDecoder. + SurfaceAllocatorService allocates SurfaceTexture(BufferQueue/GLConsumer) in a process that the SurfaceAllocatorService exists. + Then it nees to be created in a process that Compositor/WebRender exists. + SurfaceTexture is used to bind to GL texture to SurfaceTexture. android Surface just owns buffer produder. SurfaceTexture is buffer consumer(GLConsumer). + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_SurfaceAllocatorService_68.pdf + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_CodecProxy_68.pdf + https://github.com/sotaroikeda/android-diagrams/blob/master/graphics/ANativeWindow_5.0.pdf - [3] Android surface of SurfaceView needs to be delivered from Parent process to GPU process via android Binder ipc. + Compositor/WebRender create EGLSurface from the Surface and do rendering to it. + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_GeckoSession_68.pdf - [4] Needs to enable SharedSurface_SurfaceTexture in GPU process. + SharedSurface_SurfaceTexture is disabled for now even in content process(Bug 1654459) since Bug 1632249[WebGL compositing refactor (for out-of-process)]. + It is necessary for performance. - [5] How to implement video frame upload to WebGL? + GonkSurface is used for video decoding. SurfaceTexture of GonkSurface does not exist in content process. + Then current implementation allocates copyback SurfaceTexture in content process and copyback the content to content process. + If GPU process WebGL is enabled, it might not necessary to use copyback SurfaceTexture. + But SurfaceTexture has a limination. It could bind to one GL context at a time. + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_SurfaceAllocatorService_68.pdf - [6] How to handle readback of WebGL in content process? + I do not know how current GPU process WebGL handles it. It seems necessary when BasicLayerMnager is used in content process. - [7] Do we need to add support of readback of decoded video? + current gecko does not support it because of surface texture limitation. - [8] We might need to think about merging GPU process and MediaManager(:media) process for reducing memory usage.
Bug 1331109 Comment 1 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Some comments and questions around android Surface with GPU process. - [1] GeckoProcessManager needs to create GPU process + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_GeckoProcessManager_68.pdf - [2] Move SurfaceAllocatorService to GPU process + GonkSurface is used for WebGL and video decoding by MediaDecoder. + SurfaceAllocatorService allocates SurfaceTexture(BufferQueue/GLConsumer) in a process that the SurfaceAllocatorService exists. + Then it nees to be created in a process that Compositor/WebRender exists. + SurfaceTexture is used to bind to GL texture to SurfaceTexture. android Surface just owns buffer produder. SurfaceTexture is buffer consumer(GLConsumer). + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_SurfaceAllocatorService_68.pdf + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_CodecProxy_68.pdf + https://github.com/sotaroikeda/android-diagrams/blob/master/graphics/ANativeWindow_5.0.pdf - [3] Android surface of SurfaceView needs to be delivered from Parent process to GPU process via android Binder ipc. + Compositor/WebRender create EGLSurface from the Surface and do rendering to it. + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_GeckoSession_68.pdf - [4] Needs to enable SharedSurface_SurfaceTexture in GPU process. + SharedSurface_SurfaceTexture is disabled for now even in content process(Bug 1654459) since Bug 1632249[WebGL compositing refactor (for out-of-process)]. + It is necessary for performance. - [5] How to implement video frame upload to WebGL? + GonkSurface is used for video decoding. SurfaceTexture of GonkSurface does not exist in content process. + Then current implementation allocates copyback SurfaceTexture in content process and copyback the content to content process. + If GPU process WebGL is enabled, it might not necessary to use copyback SurfaceTexture. + But SurfaceTexture has a limination. It could bind to one GL context at a time. + https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_SurfaceAllocatorService_68.pdf - [6] How to handle readback of WebGL in content process? + I do not know how current GPU process WebGL handles it. It seems necessary when BasicLayerMnager is used in content process. - [7] Do we need to add support of readback of decoded video? + current gecko does not support it because of surface texture limitation. - [8] We might need to think about merging GPU process and MediaManager(:media) process for reducing memory usage.