canvas drawImage using an ImageBitmap created in a webworker as a source, is far too slow

UNCONFIRMED
Unassigned

Status

()

Core
Canvas: WebGL
P3
normal
UNCONFIRMED
3 years ago
a year ago

People

(Reporter: gordon, Unassigned)

Tracking

47 Branch
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: gfx-noted)

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.109 Safari/537.36

Steps to reproduce:

Create an ImageBitmap using "createImageBitmap".  Then once the bitmap is ready, use context.drawImage(bitmap) to render it to a canvas.


Actual results:

Large full HD resolution images still take around 40ms+.


Expected results:

Given that ImageBitmap is designed to allow fast transfer to both WebGL textures and to canvas2d, I would expect any drawImage calls using ImageBitmap objects as a source to be far faster.

On Chrome Canary, the exact same code performs it's drawImage calls in less than 1ms.

Updated

3 years ago
Component: Untriaged → Canvas: WebGL
Product: Firefox → Core

Comment 1

3 years ago
Attach a testcase, please.
Flags: needinfo?(gordon)
(Reporter)

Comment 2

3 years ago
Created attachment 8719610 [details]
testcase_1.html

Testcase 1 showing slow drawImage of ImageBitmap object that has been decoded in a webworker.
Flags: needinfo?(gordon)
(Reporter)

Comment 3

3 years ago
As requested I have attached a small testcase.  The bug is not exactly as I first suspected.  The issue only appears when the ImageBitmap object has been created inside a webworker and passed back to the main thread for rendering to a canvas.

If the ImageBitmap is created in the main thread in the standard manner then a 2048x2048 jpeg takes less than 0.5ms to drawImage to a canvas, exactly as one would expect.  If it is created inside a webworker and then sent back to the main thread (choose any local jpeg in my testcase) then it takes around 40-50ms on my 5 year old laptop.

Chrome Canary on the other hand, is always sub regardless of where the ImageBitmap was created.

It's almost as though the image is have to decode itself again on the main thread.
(Reporter)

Updated

3 years ago
Summary: canvas drawImage using ImageBitmap as a source is far too slow → canvas drawImage using an ImageBitmap created in a webworker as a source, is far too slow

Comment 4

3 years ago
An ImageBitmap stores its underlying data in a layers::Image object and while an ImageBitmap is going to be drawn onto a canvas element, we transfer the layers::Image object to a gfx::SourceSurface object which is then be drawn to the canvas element's drawTarget. 

In Windows, an blob is decoded into a layers::SourceSurfaceImage (a subclass of layers::Image) which contains an gfx::SourceSurfaceD2D1 (a subclass of gfx::SourceSurface); and a canvas element's drwaTarget object is in type of gfx::DrawTargetD2D1. Draw a gfx::SourceSurfaceD2D1 onto a gfx::DrawTargetD2D1 is very efficient.

The problem is that while transferring/structured-cloning an ImageBitmap, its underlying data is first transferred into a generic gfx::DataSourceSurface and then, in the target thread, we create a new layers::SourceSurfaceImage to contain the generic gfx::DataSourceSurface. Draw a generic gfx::DataSourceSurface onto a gfx::DrawTargetD2D1 needs an extra optimization step which is costly. 

The solution is, in short, making all subclasses of gfx::SourceSurface to be thread-safe and then we don't need a middle generic gfx::DataSourceSurface to transfer/structured-clone an ImageBitmap, however, this is still under development. 

@Morris, correct me if I was wrong.
Flags: needinfo?(mtseng)
I think you are correct! Great explanation.
Flags: needinfo?(mtseng)

Updated

2 years ago
Whiteboard: gfx-noted
You need to log in before you can comment on or make changes to this bug.