1544642 - Using the video frames as a texture on WebGL does not work

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect)

Description

[Sotaro Ikeda [:sotaro]]

The following WebGL does not work on android, since GeckoSurfaceTexture could bind to only one GL context at once. With WebRender, GeckoSurfaceTexture is soon bounded to sharedGL. It prevent from usage by WebGL.

https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/Tutorial/Animating_textures_in_WebGL

Comment 1

[Sotaro Ikeda [:sotaro]]

I saw the following error log.

04-16 12:19:42.819 29455 29747 E GLConsumer: [SurfaceTexture-0-29455-44] attachToContext: GLConsumer is already attached to a context
04-16 12:19:42.819 29455 29747 W System.err: java.lang.RuntimeException: Error during attachToGLContext (see logcat for details)
04-16 12:19:42.819 29455 29747 W System.err: at android.graphics.SurfaceTexture.attachToGLContext(SurfaceTexture.java:290)
04-16 12:19:42.819 29455 29747 W System.err: at org.mozilla.gecko.gfx.GeckoSurfaceTexture.attachToGLContext(GeckoSurfaceTexture.java:90)
04-16 12:19:42.819 29455 29747 W System.err: at org.mozilla.gecko.gfx.GeckoSurfaceTexture$NativeGLBlitHelper.blit(Native Method)
04-16 12:19:42.819 29455 29747 W System.err: at org.mozilla.gecko.gfx.GeckoSurfaceTexture.takeSnapshot(GeckoSurfaceTexture.java:304)
04-16 12:19:42.819 29455 29747 W System.err: at org.mozilla.gecko.gfx.SurfaceAllocatorService$1.sync(SurfaceAllocatorService.java:54)
04-16 12:19:42.819 29455 29747 W System.err: at org.mozilla.gecko.gfx.ISurfaceAllocator$Stub.onTransact(ISurfaceAllocator.java:93)
04-16 12:19:42.819 29455 29747 W System.err: at android.os.Binder.execTransact(Binder.java:565)

Comment 2

[Sotaro Ikeda [:sotaro]]

CompositorOGL does not cause the problem in this case, but CompositorOGL also does not handle well about contention between CompositorOGL and WebGL. For now, this bug is going to handle the problem same level as to CompositorOGL.

Comment 3

[Sotaro Ikeda [:sotaro]]

The contention happens when SurfaceTextureImage is blit by GLBlitHelper::BlitImageToFramebuffer(). SurfaceTextureImage is created only for SurfaceTexture of RemoteDataDecoder. Then the contention happen only for SurfaceTexture of video for now.

https://searchfox.org/mozilla-central/source/gfx/gl/GLBlitHelper.cpp#679

Comment 4

[Sotaro Ikeda [:sotaro]]

Attachment: Bug 1544642 - Do not bind SurfaceTexture of video to gl context if it is used by WebGL

Comment 5

[Sotaro Ikeda [:sotaro]]

The following diagram shows how video's SurfaceTexture is consumed in content process.
https://github.com/sotaroikeda/firefox-diagrams/blob/master/mobile/mobile_SurfaceAllocatorService_68.pdf

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1620e8962cab
Do not bind SurfaceTexture of video to gl context if it is used by WebGL r=nical

Comment 7

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/1620e8962cab