724482 - repeated drawImage of scaled images is insanely slow

Status: NEW

(Core :: Graphics: Canvas2D, defect)

Description

[Andreas Gal :gal]

Games often use canvas, drawImage and requestAnimationFrame for animations. If a scale factor is applied to the canvas or directly via the drawImage call, we re-scale the image every single time. This completely kills performance.

An easy fix is to cache the scaled image in CanvasImageCache, which already caches the unscaled surface. Instead, we should determine the final size of the image (applying all the transforms on the current context plus whatever scaling the arguments of drawImage result in), scale that image into a new surface, cache that surface, and then blit the surface onto the canvas.

The attached test case creates a png and then uses 100 invocations of drawImage with the same image and without and with scaling. The slowdown is around 100x on my mac.

Comment 1

[Andreas Gal :gal]

Attachment: canvas scaling test case

Comment 2

[Boris Zbarsky [:bzbarsky]]

What do the numbers look like on Windows with d2d, where we do the scaling on the GPU iirc?

It might still make sense to do the cache, but if it's just working around lack of 2d content acceleration across the board, that's worth documenting somewhere.

Comment 3

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Partly, yes.  However for downscaling it's likely more power efficient to cache scaling.  Upscaling is harder to guess.

Comment 4

[Boris Zbarsky [:bzbarsky]]

Yeah, upscaling is definitely tougher, esp because you have to use more memory for the caching.

Comment 5

[Robert O'Callahan (:roc) (email my personal email if necessary)]

How is this different from bug 724488?

Comment 6

[Andreas Gal :gal]

It can be merged with bug 724488, depending on where you want to cache. If you want to use the CanvasImageCache, its a separate bug. If you want to use image layers caching, its a separate bug. If you want to stick a cache into cairo or azure (both I guess for now), then its the same bug.

Comment 7

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Having some caching logic separated across canvas and HTML/CSS could let us make better guesses about usage.  But I agree with comment 6, the underlying mechanism could be the same.

Comment 8

[Andreas Gal :gal]

Attachment: patch

Draft patch. Not working right yet.

Comment 9

[Grant Galitz]

This is a VERY funny thing, because Google Chrome has a problem opposite to this. drawImage is fast in Google Chrome, while scaling with CSS is non-accelerated in Google Chrome. Firefox can scale super-fast by scaling the canvas with CSS rather than with canvas2d, while drawImage is super slow. The problem is you have to do a dirty UA detection or pre-flight for the fast-paths specific to each browser. See https://github.com/grantgalitz/GameBoy-Online/commit/2ac2902e08f8c7115285bc3787eb4202977e8bae where I have to check for moz by mozRequestAnimationFrame, so that the version faster for firefox can be taken by firefox.