729385 - Canvas to WebGL texture copies are slow

Status: NEW

(Core :: Graphics: CanvasWebGL, defect, P3)

Description

[Bill Baxter]

Attachment: slowCanvasToTex.zip

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.33 Safari/535.19

Steps to reproduce:

The attached sample does some Canvas2d drawing and then passes the canvas to texSubImage2D.   It times how long the texSubImage2D call takes and reports to the console the maximum time required out of each batch of 100 calls.
( Demo also up at http://www.billbaxter.com/slowCanvasToTex )


Actual results:

On Windows w/ an "Intel HD Core i5" GPU I'm seeing between 6-17 msec.



Expected results:

Ideally the copy should not take so long because it's only uploading a 64x64 square of texture.  If you slightly modify the sample to do
    var imgData = context.getImageData(0,0,64,64);
and then pass imgData to texSubImage2D, then the time for texSubImage2D is only 1ms.  So that suggests the majority of the time is related to some work done in the readback operation.

Comment 1

[Boris Zbarsky [:bzbarsky]]

Bas, Benoit, could you take a look?

Comment 2

[Bas Schouten (:bas.schouten)]

If this is on windows 7 with D2D this is probably just readback and then uploading. If we're on angle we could implement interop, but we know that interop often functions poorly, so if we want to do this a decision needs to be made that weighs the advantages against the risk and required investment.

Comment 3

[Benoit Jacob [:bjacob] (mostly away)]

Canvas-to-texture or image-to-texture uploads are currently slow in all browsers, because no browser currently tries to optimize them by avoiding the readback and just reusing the existing texture. That was actually a major discussion point in the WebGL WG recently, and I know that Chrome folk are working on canvas-to-texture uploads right now.

Bill, are you saying that canvas-to-texture uploads are even significantly slower in Firefox than Chrome? (not counting Chrome Canary if work on that has just landed).

Anyway, I agree that when using GPU layers, canvas-to-texture (and image-to-texture) uploads should be fast for the reason you say.

Bas, I suppose that in comment 2 you're talking about interop between D3D9 and D3D10 surfaces, that's true but just an implementation detail, in the worst case we could start with platforms where it's a no-brainer (Mac, Win with D3D9 layers, etc) and get back to D3D10 once ANGLE can do D3D10 or D3D11 rendering.

Comment 4

[Bill Baxter]

The same code is printing 1's in all recent versions of Chrome I tried (17, 18, 19).  WebKit had a similar slowness in canvas->texture copies previously, but it seems better now (https://bugs.webkit.org/show_bug.cgi?id=44339).

Comment 5

[Benoit Jacob [:bjacob] (mostly away)]

Bug 697443 is our tracking bug for MapsGL perf bugs.

Comment 7

[vinay karanam]

the functionality is not at all expressed

Comment 8

[Kelsey Gilbert [:jgilbert]]

We should really fix this.

Comment 10

[Rik Cabanier]

(In reply to Benoit Jacob [:bjacob] from comment #3)
> Canvas-to-texture or image-to-texture uploads are currently slow in all
> browsers, because no browser currently tries to optimize them by avoiding
> the readback and just reusing the existing texture. That was actually a
> major discussion point in the WebGL WG recently, and I know that Chrome folk
> are working on canvas-to-texture uploads right now.

This is no longer true. Chrome is optimizing this case and when possible, will reuse the texture that is already on the GPU.
This makes interchange between Canvas 2D and WebGL very fast.

Comment 11

[Benoit Jacob [:bjacob] (mostly away)]

It won't be "very fast", because drawImage or texImage2D are the wrong API for this, being inherently mean a synchronization point between script execution (on the CPU) and graphics rendering (on the GPU). "Very fast" would be to let canvases be inputs to streams as in http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/

But yes, we should still do that, it will take us from "very slow" to "not that slow".

Comment 12

[Benoit Jacob [:bjacob] (mostly away)]

Oh, disregard comment 11 --- this just needs to be a GPU-to-GPU synchronizatin point, a fence sync. I don't know what I was thinking.

Comment 13

[Rik Cabanier]

(In reply to Benoit Jacob [:bjacob] from comment #12)
> Oh, disregard comment 11 --- this just needs to be a GPU-to-GPU
> synchronizatin point, a fence sync. I don't know what I was thinking.

:-)
It would be great to have this for Firefox too. Since FF doesn't do deferred canvas rendering like Chrome, this might be fairly limited in scope.

Comment 14

[Kelsey Gilbert [:jgilbert]]

(In reply to Rik Cabanier from comment #13)
> (In reply to Benoit Jacob [:bjacob] from comment #12)
> > Oh, disregard comment 11 --- this just needs to be a GPU-to-GPU
> > synchronizatin point, a fence sync. I don't know what I was thinking.
> 
> :-)
> It would be great to have this for Firefox too. Since FF doesn't do deferred
> canvas rendering like Chrome, this might be fairly limited in scope.

What do you mean here? I'm pretty sure a couple of our canvas2d backends do this.

Comment 15

[Rik Cabanier]

(In reply to Jeff Gilbert [:jgilbert] from comment #14)
> (In reply to Rik Cabanier from comment #13)
> > (In reply to Benoit Jacob [:bjacob] from comment #12)
> > > Oh, disregard comment 11 --- this just needs to be a GPU-to-GPU
> > > synchronizatin point, a fence sync. I don't know what I was thinking.
> > 
> > :-)
> > It would be great to have this for Firefox too. Since FF doesn't do deferred
> > canvas rendering like Chrome, this might be fairly limited in scope.
> 
> What do you mean here? I'm pretty sure a couple of our canvas2d backends do
> this.

Do what? deferred rendering or reusing textures between canvas 2d and WebGL?
Looking at the canvas 2d code, it looks like immediate rendering

Comment 16

[Kelsey Gilbert [:jgilbert]]

(In reply to Rik Cabanier from comment #15)
> (In reply to Jeff Gilbert [:jgilbert] from comment #14)
> > (In reply to Rik Cabanier from comment #13)
> > > (In reply to Benoit Jacob [:bjacob] from comment #12)
> > > > Oh, disregard comment 11 --- this just needs to be a GPU-to-GPU
> > > > synchronizatin point, a fence sync. I don't know what I was thinking.
> > > 
> > > :-)
> > > It would be great to have this for Firefox too. Since FF doesn't do deferred
> > > canvas rendering like Chrome, this might be fairly limited in scope.
> > 
> > What do you mean here? I'm pretty sure a couple of our canvas2d backends do
> > this.
> 
> Do what? deferred rendering or reusing textures between canvas 2d and WebGL?
> Looking at the canvas 2d code, it looks like immediate rendering

Yeah, it sounds like we don't do this yet.

Comment 17

[Chris Peterson [:cpeterson]]

Shumway has two compositing backends: Canvas2D and WebGL. We could not do WebGL compositing, but that would be very slow. We want to copy Canvas2D texture to WebGL context without reading back from GPU.

Comment 18

[Chiajung Hung [:chiajung]]

I am working on a patch for bug 1001069. However, I think bug 1001069 should handle 2d canvas part only, I would like to update the status of WebGL related part in this bug.

Comment 19

[Chiajung Hung [:chiajung]]

I found I am confused with these bugs...
It seems this bug deals with the WebGL::Tex{Sub}Image2D(<canvas>) API
And Bug 952539 deals with the WebGL::Tex{Sub}Image2D(ImageData) API

Anyway, the solution is almost the same. So let me update the ImageData part in Bug 952539, and full canvas case here later on.