1081887 - 2D canvas method .putImageData() is unreasonably slow for dynamic updates.

Status: RESOLVED FIXED

(Core :: Graphics: Canvas2D, defect)

Description

[Jukka Jylänki]

This is similar to https://bugzilla.mozilla.org/show_bug.cgi?id=1001069 , but the discussions relate to a different optimization, so probably not a duplicate of that.

Emscripten SDL-based applications use 2D canvases for the on-screen surface. Typical blit code looks like follows: https://github.com/kripken/emscripten/blob/incoming/src/library_sdl.js#L1511 , i.e. in the fast path:

    var data = surfData.image.data;
    var data32 = new Uint32Array(data.buffer);
    data32.set(HEAP32.subarray(src, src + data32.length));
    surfData.ctx.putImageData(surfData.image, 0, 0);

Profiling the performance of applications that use such a method, more than 20% of per-frame execution time is spent in the single function

    mozilla::dom::CanvasRenderingContext2D::PutImageData_explicit(int, int, unsigned int, unsigned int, mozilla::dom::TypedArray<unsigned char, &(js::UnwrapUint8ClampedArray(JSObject*)), &(JS_GetUint8ClampedArrayData(JSObject*, JS::AutoCheckCannotGC const&)), &(js::GetUint8ClampedArrayLengthAndData(JSObject*, unsigned int*, unsigned char**)), &(JS_NewUint8ClampedArray(JSContext*, unsigned int))>*, bool, int, int, int, int)

which looks like this: http://dxr.mozilla.org/mozilla-central/source/dom/canvas/CanvasRenderingContext2D.cpp#4650

Looking at the code, some optimization opportunities come to mind:
  - The function unconditionally allocates a new gfxImageSurface every time on line 4713. This looks unnecessary, as the destination to blit to already exists.
  - There is a slow per-byte loop to convert image data to premultiplied alpha. Using SIMD would speed it up an order of magnitude, or even better, finding a way not to need to convert the data to premultiplied form at all would be best. Also, it looks like the conversion loop always converts the whole image, and not just the dirty rectangle, so even if a very small part of the image is dirty, the whole image gets premultiply-converted.
  - Line 4751 looks like another unconditional new surface allocation: mTarget->CreateSourceSurfaceFromData(imgsurf->Data(), IntSize(w, h), imgsurf->Stride(), SurfaceFormat::B8G8R8A8); and possibly a temporary image blit. Again, this surface clone is of full size, and not only of the size of the dirty rectangle.
  - Line 4761 finally performs the blit: mTarget->CopySurface(...). If premultiplied alpha conversion is needed, it would probably be fastest to fuse to this copy so that memory access footprint remains low.

Comment 1

[Jukka Jylänki]

The Voxatron demo snippet on Firefox main page suffers heavily from this performance impact.

Comment 2

[Milan Sreckovic [:milan] (needinfo for best results)]

Nice analysis, Jukka.

Just to comment on this issue vs. bug 1001069 - in the later, we're really looking at the scenario where we're only getting/setting a subset of the surface, and want the performance to be related to the size of the data that is being read/written, rather than the size of the underlying surface.

In this bug, we're updating the whole thing, so it does stand on its own, and is a different issue than that in bug 1001069.

Comment 3

[Lee Salzman [:lsalzman]]

Jukka, given recent improvements in putImageData, does this issue still stand? There have been SIMD improvements to the premul, some removal of copies, etc.

Comment 4

[Jukka Jylänki]

Hey, very nice! Taking a look at a profile of the Voxatron demo, I see that it is now taking a SSE2 path and the overhead is reduced to ~4% from the original ~20%.

I've had a TODO for quite a while to create a stress test of these SDL canvas putImageData() style of renderers which are common for Emscripten based 2D applications to use, the Voxatron demo is now on a bit more on the simpler end. If there are any new perf bottlenecks surfaced with such a stress test, I'll open a new item, but for Voxatron's case, this is certainly considered to be fixed. Great job!