Open Bug 1248392 Opened 8 years ago Updated 2 years ago

canvas drawImage using an ImageBitmap created in a webworker as a source, is far too slow

Categories

(Core :: Graphics: Canvas2D, defect, P3)

47 Branch
defect

Tracking

()

People

(Reporter: gordon, Unassigned)

Details

(Keywords: perf, Whiteboard: gfx-noted)

Attachments

(1 file)

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.109 Safari/537.36

Steps to reproduce:

Create an ImageBitmap using "createImageBitmap".  Then once the bitmap is ready, use context.drawImage(bitmap) to render it to a canvas.


Actual results:

Large full HD resolution images still take around 40ms+.


Expected results:

Given that ImageBitmap is designed to allow fast transfer to both WebGL textures and to canvas2d, I would expect any drawImage calls using ImageBitmap objects as a source to be far faster.

On Chrome Canary, the exact same code performs it's drawImage calls in less than 1ms.
Component: Untriaged → Canvas: WebGL
Product: Firefox → Core
Attach a testcase, please.
Flags: needinfo?(gordon)
Attached file testcase_1.html
Testcase 1 showing slow drawImage of ImageBitmap object that has been decoded in a webworker.
Flags: needinfo?(gordon)
As requested I have attached a small testcase.  The bug is not exactly as I first suspected.  The issue only appears when the ImageBitmap object has been created inside a webworker and passed back to the main thread for rendering to a canvas.

If the ImageBitmap is created in the main thread in the standard manner then a 2048x2048 jpeg takes less than 0.5ms to drawImage to a canvas, exactly as one would expect.  If it is created inside a webworker and then sent back to the main thread (choose any local jpeg in my testcase) then it takes around 40-50ms on my 5 year old laptop.

Chrome Canary on the other hand, is always sub regardless of where the ImageBitmap was created.

It's almost as though the image is have to decode itself again on the main thread.
Summary: canvas drawImage using ImageBitmap as a source is far too slow → canvas drawImage using an ImageBitmap created in a webworker as a source, is far too slow
An ImageBitmap stores its underlying data in a layers::Image object and while an ImageBitmap is going to be drawn onto a canvas element, we transfer the layers::Image object to a gfx::SourceSurface object which is then be drawn to the canvas element's drawTarget. 

In Windows, an blob is decoded into a layers::SourceSurfaceImage (a subclass of layers::Image) which contains an gfx::SourceSurfaceD2D1 (a subclass of gfx::SourceSurface); and a canvas element's drwaTarget object is in type of gfx::DrawTargetD2D1. Draw a gfx::SourceSurfaceD2D1 onto a gfx::DrawTargetD2D1 is very efficient.

The problem is that while transferring/structured-cloning an ImageBitmap, its underlying data is first transferred into a generic gfx::DataSourceSurface and then, in the target thread, we create a new layers::SourceSurfaceImage to contain the generic gfx::DataSourceSurface. Draw a generic gfx::DataSourceSurface onto a gfx::DrawTargetD2D1 needs an extra optimization step which is costly. 

The solution is, in short, making all subclasses of gfx::SourceSurface to be thread-safe and then we don't need a middle generic gfx::DataSourceSurface to transfer/structured-clone an ImageBitmap, however, this is still under development. 

@Morris, correct me if I was wrong.
Flags: needinfo?(mtseng)
I think you are correct! Great explanation.
Flags: needinfo?(mtseng)
Whiteboard: gfx-noted
Status: UNCONFIRMED → NEW
Ever confirmed: true
Component: Canvas: WebGL → Canvas: 2D
Priority: P3 → --
Keywords: perf
Priority: -- → P3
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: