1655101 - DelightVR video playback failing in android (webgl.texImage(RGB8,video))

Status: RESOLVED FIXED

(Core :: Graphics: CanvasWebGL, defect)

Description

[Imanol Fernandez]

STR with remote WebGL enabled:

• Open https://delight-vr.com/video-player-module/
• Click the Play button on the Surf video at the bottom of the page
• Black screen while playing the video in WebXR

I saw these erros in logcat

2020-07-24 19:02:15.937 24842-24912/org.mozilla.vrbrowser.dev I/AdrenoGLES-0: Driver Path                      : /system/vendor/lib64/egl/libGLESv2_adreno.so
2020-07-24 19:02:16.541 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.541 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.541 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.541 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.541 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.541 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.541 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.542 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.542 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.542 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.542 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.542 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.542 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.542 24842-24912/org.mozilla.vrbrowser.dev E/libEGL: call to OpenGL ES API with no current context (logged once per thread)
2020-07-24 19:02:16.595 24842-24901/org.mozilla.vrbrowser.dev I/Gecko: Can't find symbol 'eglDupNativeFenceFDANDROID'.

Comment 1

[Imanol Fernandez]

Three.js VR video playback also shows a black screen: https://github.com/MozillaReality/FirefoxReality/issues/3727

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1607940

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:jgilbert, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 4

[u597032]

Firefox reality is no longer supported
https://blog.mozilla.org/en/mozilla/news/update-on-firefox-reality/

Comment 5

[Emilio Cobos Álvarez (:emilio)]

This also applies to FF/Android

Comment 6

[Sergio Villar]

(In reply to Emilio Cobos Álvarez (:emilio) from comment #5)

This also applies to FF/Android

Indeed, thanks Emilio for bringing this back to life. Also note that, contrary to what is said in the 2nd comment by Imanol, it happens also with in-process WebGL. In particular, in Wolvic, we have to disable the GPU process because WebXR is broken in that case. We have the very same issues that FF/Android with those videos.

Related to this, https://github.com/playcanvas/engine/issues/4204. Also similar info in this other issue https://github.com/Igalia/wolvic/issues/370

Comment 7

[Jamie Nicol [:jnicol]]

I hadn't seen this before. The issue is that the texture format is GL_RGB, meaning we prevent the fast-path blit here, meaning we attempt to take the slow path here. But the slow path fails on Android because we do not implement GetAsSourceSurface() here.

See bug 1769747 for falling off the fast path on other platforms too due to the RGB format, though the slow fallback works so it's just a performance issue on other platforms.

Implementing GetAsSourceSurface() would fix this, albeit with poor performance. And would also fix the similar canvas2d issue (bug 1526207). But as the comment alludes to, doing so would mean we could no longer render the video element. Perhaps that would be preferable.

Kelsey, my reading of the spec is that GL_RGB is color-renderable on GLES 3.0+ and desktop GL 3.0+. (And on GLES 2 we could perhaps check for GL_OES_rgb8_rgba8, though I'm not 100% if that is enough). Would a patch be accepted for allowing the blit to an RGB texture in those cases? (But not the other formats you expressed concerns about in bug 1769747)

Comment 8

[Jamie Nicol [:jnicol]]

Attachment: Bug 1655101 - Allow blit to GL_RGB texture for webgl texture uploads. r?jgilbert

On devices where GL_RGB is a color-renderable format, allow blitting
to GL_RGB textures to implement texture upload. This allows us to stay
on the fast-path when the source data is video and the target texture
format is GL_RGB. In fact, on Android this makes it work at all, as
the slow-path is not actually implemented.

Comment 9

[Sergio Villar]

(In reply to Jamie Nicol [:jnicol] from comment #8)

Created attachment 9312852 [details]
Bug 1655101 - Allow blit to GL_RGB texture for webgl texture uploads. r?jgilbert

On devices where GL_RGB is a color-renderable format, allow blitting
to GL_RGB textures to implement texture upload. This allows us to stay
on the fast-path when the source data is video and the target texture
format is GL_RGB. In fact, on Android this makes it work at all, as
the slow-path is not actually implemented.

I can confirm that this makes Wolvic really happy :-). Video is now properly rendered in DelightXR sites

Comment 10

[Sergio Villar]

Note that there are still some issues. For example when playing a video in https://mindvr.com, sometimes it happens that the video is not rendered and console spits many WebGL warnings like the following ones

XRWebGLLayer doesn't support no alpha value. Alpha will be enabled. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:52:30120
XRWebGLLayer doesn't support separate depth or stencil buffers. They will be enabled together. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:52:30120
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: drawElementsInstanced: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. 2 dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
WebGL warning: clear: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:17:27463
WebGL warning: drawElementsInstanced: Framebuffer must be complete. dl8-c7f4d76df2a59bf0469e2a020e65ed8fcc50640e.js:18:7855
After reporting 32, no further warnings will be reported for this WebGL context.
``

Comment 11

[Jamie Nicol [:jnicol]]

Can you please file a separate bug about that? Thanks.

Comment 12

[Sergio Villar]

(In reply to Jamie Nicol [:jnicol] from comment #11)

Can you please file a separate bug about that? Thanks.

Sure, done

Comment 14

[Sergio Villar]

:jnicol: (In reply to Jamie Nicol [:jnicol] from comment #8)

Created attachment 9312852 [details]
Bug 1655101 - Allow blit to GL_RGB texture for webgl texture uploads. r?jgilbert

On devices where GL_RGB is a color-renderable format, allow blitting
to GL_RGB textures to implement texture upload. This allows us to stay
on the fast-path when the source data is video and the target texture
format is GL_RGB. In fact, on Android this makes it work at all, as
the slow-path is not actually implemented.

There is another issue in the patch. During a single execution, the first time I try to play any delightxr powered video, it isn't played (in immersive mode). I mean, I can hear audio after a while, but video is not played. However it does always work for any video after that first attempt. Does it make sense? Rings any bell?

Comment 15

[Jamie Nicol [:jnicol]]

No idea, sorry. Sounds like it's a separate bug rather than an issue with the above patch, though?

Comment 16

[Sergio Villar]

Is this going to land?

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b09d48d2b473
Allow blit to GL_RGB texture for webgl texture uploads. r=aosmond

Comment 18

[Sandor Molnar[:smolnar]]

Backed out for causing mochitest webgl failures on dom/canvas/TexUnpackBlob.cpp

Backout link: https://hg.mozilla.org/integration/autoland/rev/4fe7bea775a57b82c36a36f36b7d68b31b2c9d75

Push with failures

Failure log -> Assertion failure: false (status == LOCAL_GL_FRAMEBUFFER_COMPLETE), at /builds/worker/checkouts/gecko/dom/canvas/TexUnpackBlob.cpp

Comment 19

[Jamie Nicol [:jnicol]]

The test failure is because we attach a GL_SRBG8 format texture to the framebuffer, which angle says is always incomplete due to this.

My intuition tells me that BlitPreventReason needs to check the texture's internal format rather than pi.format. But this is a bit beyond my knowledge. Kelsey, does that sound right?

Comment 20

[Kelsey Gilbert [:jgilbert]]

Attachment: Bug 1655101 - Allow webgl.texImage gpu-blit for RGB8 if supported.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7aa58b8ef68a
Allow webgl.texImage gpu-blit for RGB8 if supported. r=gfx-reviewers,aosmond

Comment 22

[Cosmin Sabou [:CosminS]]

Backed out for causing bustages on dom/WebGLIpdl.h.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=7aa58b8ef68a5fac39a8cd62dcf462f8ca309e5c&selectedTaskRun=YSBEetRET4u1-YikP2yn3A.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=438347678&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/ac99cb1291098db9161c56ba4e184dd55edb0fb2

Comment 23

[Kelsey Gilbert [:jgilbert]]

Attachment: Bug 1655101 - AssertTiedFieldsAreExhaustive<T>() gives better compiler errors.

A templated class FieldDebugInfoT is used, because compilers will print
the template parameters of classes involved in error messages, so we can
use this to print a number of pieces of info, which will pinpoint where
the issue is.

static_assert with constexpr formatting is coming in C++26. :')

Comment 24

[Kelsey Gilbert [:jgilbert]]

Attachment: Bug 1655101 - Padded<T,PaddedSize> to paper over mingw-32's smaller sizeof(std::string).

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8c1d5e0757d2
Allow webgl.texImage gpu-blit for RGB8 if supported. r=gfx-reviewers,aosmond
https://hg.mozilla.org/integration/autoland/rev/260f3f01f197
AssertTiedFieldsAreExhaustive<T>() gives better compiler errors. r=gfx-reviewers,aosmond
https://hg.mozilla.org/integration/autoland/rev/f0526169a716
Padded<T,PaddedSize> to paper over mingw-32's smaller sizeof(std::string). r=gfx-reviewers,aosmond

Comment 26

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/8c1d5e0757d2
https://hg.mozilla.org/mozilla-central/rev/260f3f01f197
https://hg.mozilla.org/mozilla-central/rev/f0526169a716

Comment 27

[Acasandrei Beatrice (needinfo me)]

(In reply to Pulsebot from comment #25)

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8c1d5e0757d2
Allow webgl.texImage gpu-blit for RGB8 if supported. r=gfx-reviewers,aosmond
https://hg.mozilla.org/integration/autoland/rev/260f3f01f197
AssertTiedFieldsAreExhaustive<T>() gives better compiler errors.
r=gfx-reviewers,aosmond
https://hg.mozilla.org/integration/autoland/rev/f0526169a716
Padded<T,PaddedSize> to paper over mingw-32's smaller sizeof(std::string).
r=gfx-reviewers,aosmond

== Change summary for alert #40939 (as of Wed, 10 Jan 2024 06:13:40 GMT) ==

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
24%	espn FirstVisualChange	linux1804-64-shippable-qr	cold fission webrender	894.76 -> 675.74	Before/After
12%	espn PerceptualSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	1,095.18 -> 963.07	Before/After
10%	espn PerceptualSpeedIndex	linux1804-64-shippable-qr	cold fission webrender	1,048.69 -> 946.04	Before/After

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=40939