1024566 - Provide an API to decode image data synchronously

Status: NEW

(Core :: DOM: Workers, enhancement, P5)

Description

[Tobias Schneider [:tobytailor]]

In our efforts to move most parts of Shumway into Workers, decoding image data to read pixel values is one of the few remaining parts where we would have to block execution in the worker (if even possible someday) and waiting for asynchronous image loading and decoding on the main thread to be finished. Not to mention the overhead when sending image data, creating a temporary canvas to read pixels from and eventually sending back the resulting pixel buffer. Therefore we would like to propose a way to decode images synchronously in a Worker when the encoded image data is already available as a Blob or Data URI. Since the ImageData constructor is now exposed to Workers, one possible solution to decode a Blob containing image file data could work as shown in the following example:

...
var imageData = new ImageData(imgWidth, imgHeight);
imageData.decodeBlob(encodedImageBlob);
var decodedPixelData = imageData.data;
...

Comment 1

[Till Schneidereit [:till]]

Apart from Shumway, my guess is that this is also interesting for games. Once we have WebGL on workers[1], decoding textures delivered as JPGs will be a main-thread bottleneck.


[1]: bug 709490

Comment 2

[Michael Bebenita [:mbx]]

What should the proposed API do if the specified |imgWidth| and |imgHeight| don't match the dimensions of the decoded image.

A more straight forward API would be:

var imageData = new ImageData(blob | typedarray);

If the image is successfully decoded, the dimensions of the |imageData| will match those of the decoded image.

It would also be nice to decode the image to a specified scale:

var imageData = new ImageData(blob | typedarray, filteringMode, width, optional height);

The decoded image will now be scaled to match the specified dimensions. The advantage here is that you save some memory by not keeping around the original decoded image.

Comment 3

[Tobias Schneider [:tobytailor]]

There is already an overloaded ImageData constructor using a similar signature (taking an Uint8ClampedArray containing RGBA data as its first argument and a width and height argument to specify the image dimensions).
We could introduce an optional argument specifying the type of data in the passed Uint8ClampedArray (RGBA, JPEG, PNG ...) that would also change semantics of the dimension arguments to handle scaling.