775706 - Use SurfaceTexture for WebGL

Status: RESOLVED WONTFIX

(Firefox for Android Graveyard :: Toolbar, defect)

Description

[James Willcox (:snorp) ([e-mail]) (he/him)]

At least in theory, backing our WebGL contexts with a SurfaceTexture could greatly improve performance due to the lack of copying necessary. We should investigate it.

Comment 1

[James Willcox (:snorp) ([e-mail]) (he/him)]

Attachment: Use SurfaceTexture for WebGL on Android

Comment 2

[Kelsey Gilbert [:jgilbert]]

(In reply to James Willcox (:snorp) ([e-mail]) from comment #0)
> At least in theory, backing our WebGL contexts with a SurfaceTexture could
> greatly improve performance due to the lack of copying necessary. We should
> investigate it.

With the EGLImage stuff that landed recently, this shouldn't give any great improvement for performance. We should wait on bug 716859 for the refactor of this area.

Comment 3

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

In theory it should get rid of CopyTexImage, would it not?  That should be noticeable, but I don't see any significant difference unfortunately.

Also, there's a bug -- load http://conduit.bitops.com/~vladimir/misc/clear-varying-color-noalpha-pm.html and then when it's done, pan the window around.  The background (webgl canvas) color will keep changing for a while, as if there were pending updates.  That's a little weird. :)

Comment 4

[Kelsey Gilbert [:jgilbert]]

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #3)
> In theory it should get rid of CopyTexImage, would it not?  That should be
> noticeable, but I don't see any significant difference unfortunately.
> 
> Also, there's a bug -- load
> http://conduit.bitops.com/~vladimir/misc/clear-varying-color-noalpha-pm.html
> and then when it's done, pan the window around.  The background (webgl
> canvas) color will keep changing for a while, as if there were pending
> updates.  That's a little weird. :)

We're getting a generic and relatively simple interface for this soon. This will provide no-copy webgl where possible. I would much prefer to wait about a week for this to go in.

Comment 5

[James Willcox (:snorp) ([e-mail]) (he/him)]

On Galaxy Nexus there doesn't seem to be much difference between SurfaceTexture and the current implementation. However, on the HTC One X (which is S4/Adreno), there is a pretty dramatic difference. I think we may want SurfaceTexture eventually just to work around buggy CopyTexSubImage2D bustedness.

Comment 6

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Interesting! I was also trying on Galaxy Nexus; I'll try on my Tegra 3 One X at some point too.

Comment 7

[James Willcox (:snorp) ([e-mail]) (he/him)]

Attachment: Another WIP, now with fences

Not sure if I'm really using the fence stuff properly...

Comment 8

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

SwapBuffers discussion from irc:

14:04 <@vlad> snorp: so UpdateSharedHandle is called in
              BasicShadowableCanvasLayer::Paint
14:05 <@vlad> which should be the right time, since that'll be called when we're
              actually about to "composite" (that is, send the transaction to
              the compositor)
14:05 <@vlad> but the problem you're having is that the main thread can then
              swap again, if it's already ready to queue another frame before
              the compositor is ready for it
14:05 <@vlad> (I think.)
14:05 <@vlad> jgilbert's patch doesn't ever swap like that, it builds up a queue
              of frames and only reuses a frame/texture when the compositor is
              done with it

So to use SurfaceTexture, we have to end up blocking on the compositor being done with the frame.  Or to throw it away if it's not ready for it, which is an interesting idea that would require some webgl spec changes (an additonal context creation flag, basically telling the system that rendering will be continuous, so that it can skip compositing frames after it's needed).

Comment 9

[James Willcox (:snorp) ([e-mail]) (he/him)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/fd9019f20ca4

Comment 10

[James Willcox (:snorp) ([e-mail]) (he/him)]

Ignore the above, wrong bug.

Comment 11

[James Willcox (:snorp) ([e-mail]) (he/him)]

Attachment: Use SurfaceTexture for WebGL on Android when able

Comment 12

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

This looks great -- let's do it, but please make preserveDrawingBuffer=true continue to work (by just bailing out of the SurfaceTexture path, most likely I think).

Comment 13

[James Willcox (:snorp) ([e-mail]) (he/him)]

Attachment: Use SurfaceTexture for WebGL on Android when able

Comment 14

[James Willcox (:snorp) ([e-mail]) (he/him)]

Green-ish Try run here: https://tbpl.mozilla.org/?tree=Try&rev=abd9db250436

Comment 15

[James Willcox (:snorp) ([e-mail]) (he/him)]

The ContextFlags changes were a little more invasive than I initially planned, but I think it's more correct now.

Comment 16

[Kelsey Gilbert [:jgilbert]]

This seems like it'll just be immediately supplanted by bug 716859.

Comment 17

[James Willcox (:snorp) ([e-mail]) (he/him)]

This bug obsoleted by bug 716859