1537961 - Getting VR WebGL content to the screen involves too many copies on the Oculus Go in Firefox Reality

Status: RESOLVED INCOMPLETE

(Core :: WebVR, enhancement)

Description

[Markus Stange [:mstange]]

There seem to be three buffers that VR WebGL content passes through on its way to the screen on the Oculus Go:

Oculus creates an Android surface which is double buffered.
GeckoView has a queue of surfaces ("GeckoSurface") which it cycles through.
Gecko has a surface for the WebGLContext, its mDefaultFB.

Firefox Reality was assuming that Gecko was writing directly into the GeckoView surface and that we would only copy once. However, it seems there is at least one additional copy: The one from the WebGLContext's mDefaultFB to the GeckoView surface.

This copy happens in WebGLContext::BlitBackbufferToCurDriverFB(), through the following call path:

VRDisplay.submitFrame -> WebGLContext::GetVRFrame() -> WebGLContext::PresentScreenBuffer() -> WebGLContext::BlitBackbufferToCurDriverFB()

It would be good to eliminate this copy.

(At some point we'd also like to bypass the copy of the GeckoView surface to the surface that comes from Oculus. This won't work on other VR devices, though, so we should focus on WebGL's copy first.)

Comment 1

[Daosheng Mu[:daoshengmu]]

I am going to work on this.

Comment 2

[Daosheng Mu[:daoshengmu]]

It looks like GeckoSurface or GeckoSurfaceTexture between GV and FxR are not really doing copy. They will look up textureHandle if it is existing in the GeckoSurfaceTexture map, if so, it will return a refPtr. The bad thing for using GeckoSurface might be it will ask for synchronization IPCs that will lock the possibility for Oculus Go to do timewarp layers.

Actually, we have two copies in WebGL::PresentScreenBuffer(), one is blitting framebuffer from WebGLContext::mDefaultFB to the driver FB. The second one is at GLScreenBuffer::PublishFrame(), we do a copy from its front buffer to the back buffer. It looks like we can skip the first driver copy and let GLScreenBuffer copies its buffer from WebGLContext::mDefaultFB and do swap().

However, for the maximum optimization, we should just let WebGLContext::mDefaultFB copy to Oculus's timewarp layer.

[1] https://dxr.mozilla.org/mozilla-central/rev/93075ec49df3982c26873b822d762bd3d8863fad/dom/canvas/WebGLContext.cpp#1393-1405

Comment 3

[Daosheng Mu[:daoshengmu]]

Attachment: Bug 1537961 - (wip) skip WebGL driver FB copy when doing VR immersive frames.

MozReview-Commit-ID: 4tArfixXkSW

Comment 4

[Daosheng Mu[:daoshengmu]]

For WebGLContext::PresentScreenBuffer(), we did a copy in BlitBackbufferToCurDriverFB() from WebGL::mDefaultFB to gl->Screen sharedSurface to make it to be shareable cross other processes. Then, when in VR mode(), we will swap buffer from gl->Screen() to VRScreen's front buffer. We need the first copy to make sure WebGL render process can be isolated from the composition or other backend process. It sounds like reasonable. If we want to replace WebGL::mDefaultFB with a sharedSurface to make render directly to a sharedSurface, we might need to do a lot of works on stability issues.

Comment 5

[Daosheng Mu[:daoshengmu]]

We spent about 13% time for doing WebGL::mDefaultFB to VRScreen's front buffer in WebGLContext.cpp

2.02 % mozilla::WebGLContext::BlitBackbufferToCurDriverFB()
2.34 % mozilla::gl::GLScreenBuffer::Swap()
3.61 % mozilla::gl::SharedSurface_SurfaceTexture::Commit()

At GLScreenBuffer::Swap(), we will try to recycle the existing TextureClient if the screen size is the same and attach the buffer to glContext. Finally, asking glContext swap the previous buffer with this new attached buffer. For this part, we didn't generate a new buffer and didn't have a copy either, but the gl attach() / swap() function calls take a lot of time. It looks like we are waiting for GPU doing the BlitBackbufferToCurDriverFB().

Besides, according to this profile, it seems like we can't see information about GV surface and Android surface. But at least, from VRLayerChild::SubmitFrame() to gfxVRExternal, I didn't see any copy.

Comment 6

[Daosheng Mu[:daoshengmu]]

According to the recent profiling information [1], VRDisplay::SubmitFrame() only takes about 2% time of total. But, JS engine takes more than 70% time usage. It looks like the buffer copy is not so important compared with the JS side. One thing I will continue to investigate is why we do multiple VRDisplay::SubmitFrame() in a FrameRequestCallback(). It looks like we didn't meet the headset display refresh rate (60 HZ on Oculus Go), and do these multiple VRDisplay::SubmitFrame(), buffer copies in one driver fresh time.

[1] https://profiler.firefox.com/public/9841f87daac03548210128b6e57351bb68b00e5c/calltree/?globalTrackOrder=0-1&implementation=cpp&localTrackOrderByPid=2651-0~&published&thread=2&v=3

Comment 7

[u597032]

Firefox reality is no longer supported
https://blog.mozilla.org/en/mozilla/news/update-on-firefox-reality/