1164458 - Canvas context.drawImage should work with an Image loaded from a data URI even before the load event is delivered

Status: REOPENED

(Core :: Graphics: Canvas2D, defect)

Description

[Ermir Suldashi]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0
Build ID: 20150305021524

Steps to reproduce:

Step 1. Create a new Image() element and set its src attribute to a base64 PNG. Do not append this element to any element in the DOM
Step 2. Get a canvas element's 2D context, and call context.drawImage, and supply the image from step 1 to it.


Actual results:

The canvas element becomes blank.


Expected results:

The canvas element should have the image attached to it.

Workaround: attach the image element to somewhere in the screen where it is not visible. Setting the image or its container to display: none does not work, so what I did was set the position to absolute and shifted it left by -15000px;

Comment 1

[Loic]

Could you attach a testcase to the bug, please.

Comment 2

[Gingerbread Man]

Attachment: canvas.html

Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:41.0) Gecko/20100101 Firefox/41.0
Build ID: 20150512030215

All you have to do is wait for the image to load (yes, even though it's base64-encoded). Then again, IE 11, Chromium 41 and Opera 28 all load the image just fine, so maybe there would be interest in fixing this.

Comment 3

[Kelsey Gilbert [:jgilbert]]

Thoughts?

Comment 4

[Jeff Muizelaar [:jrmuizel]]

The test case is assuming that data urls will be decoded synchronously. I don't believe we treat data urls specially in this regard so there's no reason to expect this to work.

Seth or Boris do either of you remember anything about our intentions here?

Comment 5

[Seth Fowler [:seth] [:s2h]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #4)
> The test case is assuming that data urls will be decoded synchronously. I
> don't believe we treat data urls specially in this regard so there's no
> reason to expect this to work.
> 
> Seth or Boris do either of you remember anything about our intentions here?

Our intention was to support this and we have a partially completed patch in bug 1092837. However, nobody has had time to push it over the finish line.

We also want to support this for *some kinds* of blob URIs. Some blobs can't be retrieved synchronously, though.

Comment 6

[Boris Zbarsky [:bzbarsky]]

Per spec, painting an image to a canvas immediately after setting src should not work, even for a data: image, unless the image has been preloaded into the document's "ist of available images".  So if you want to change behavior here it sounds like the spec needs changing.

Comment 7

[Jeff Muizelaar [:jrmuizel]]

(In reply to Boris Zbarsky [:bz] from comment #6)
> Per spec, painting an image to a canvas immediately after setting src should
> not work, even for a data: image, unless the image has been preloaded into
> the document's "ist of available images".  So if you want to change behavior
> here it sounds like the spec needs changing.

Do you know where the spec says this?

Comment 9

[Boris Zbarsky [:bzbarsky]]

Sure.  If you look at the spec for drawImage, at <https://html.spec.whatwg.org/multipage/scripting.html#drawing-images:dom-context-2d-drawimage-4>, step 1 says to do https://html.spec.whatwg.org/multipage/scripting.html#check-the-usability-of-the-image-argument and if "bad" is returned just do nothing.

If you look at the check-the-usability steps, step 2 says to return "bad" if the arg is an HTMLImageElement that is not "fully decodable".  This is defined at https://html.spec.whatwg.org/multipage/embedded-content.html#img-good as:

  When an img element is in the completely available state and the user
  agent can decode the media data without errors, then the img element
  is said to be fully decodable.

"Completely available" is defined at https://html.spec.whatwg.org/multipage/embedded-content.html#img-all as:

  The user agent has obtained all of the image data and at least 
  the image dimensions are available.

Note that this applies to the "current request" of the image, but the spec explicitly says it uses that interchangeably with the image itself when it's not ambiguous.

OK, so now the next thing you need to look at is what state the current request of the image is actually in.  Our scenario is that we new Image(), then set its src, then immediately drawImage.  The behavior of new Image is to set the current request to a new image request.  If you read a bit down from the "#img-all" link above, you see:

  An image request is initially unavailable.

OK, so now we set .src.  This is a "relevant mutation" in the sense of https://html.spec.whatwg.org/multipage/embedded-content.html#relevant-mutations so this paragraph applies:

  A user agent that obtains images immediately must synchronously update
  the image data of an img element, with the restart animation flag set
  if so stated, whenever that element is created or has experienced 
  relevant mutations.

OK, so setting .src lands us in https://html.spec.whatwg.org/multipage/embedded-content.html#update-the-image-data and we start going through the steps.  Ignoring all the complications for srcset and picture, we land in step 5 with the "selected source" being the value assigned to .src.  We look up the corresponding absolute URL in the "list of available images".  If that finds something, we set up a new current request in the "completely available" state and are done.  If not, the rest of this algorithm (starting in step 6) happens async from the point of view of script.  So at the drawImage point we still have our initial current request, which is in the "unavailable" state.

OK, so how does something end up in the "list of available images"?  https://html.spec.whatwg.org/multipage/embedded-content.html#list-of-available-images specified this, but it's basically an attempt to spec the per-document (or in Gecko's case global) image cache UAs have.  The spec says that things can end up in the list of available images in several ways:

1)  A copy from some other document's list of available images (see <https://html.spec.whatwg.org/multipage/embedded-content.html#list-of-available-images>).

2)  Somewhere in the guts of step 14 of <https://html.spec.whatwg.org/multipage/embedded-content.html#update-the-image-data>: This is the "we got the image from the server, let's add it to our cache" case.

3)  From step 15 of https://html.spec.whatwg.org/multipage/embedded-content.html#img-environment-changes which is intended to handle srcset and the like responding to media query changes.  Note that, in any case, step 15 happens async from the point of view of this algorithm.

Comment 10

[Boris Zbarsky [:bzbarsky]]

Note that if no one is implementing the spec then we should get the spec fixed.  But then we need to either reverse-engineer people or get them to tell us what they actually implement....

Comment 11

[Boris Zbarsky [:bzbarsky]]

https://github.com/whatwg/html/issues/656 I guess.

Comment 12

[Anne (:annevk)]

It seems that is discussed there in detail with the Blink engineers. Clearing needinfo request.