Closed Bug 1629381 Opened 2 years ago Closed 5 months ago

Calling ctx.drawImage throws NS_ERROR_NOT_AVAILABLE when drawing a video on Windows 10

Categories

(Core :: Graphics, defect, P3)

74 Branch
Desktop
Windows 10
defect

Tracking

()

RESOLVED FIXED
90 Branch
Tracking Status
firefox90 --- fixed

People

(Reporter: wessel_kroos, Assigned: emilio)

References

()

Details

Attachments

(1 file)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36

Steps to reproduce:

I've caught exceptions from users, but I haven't found a way to reproduce the bug yet.

Actual results:

When calling canvasContext.drawImage(video, ..) a NS_ERROR_NOT_AVAILABLE is thrown.

This happened for multiple users on Firefox 74 for Windows 10. But I don't have a test case.
I've caught 10 exceptions over the last 30 days. (And more exceptions pre version 74, but that data has been lost.)
Based on the HTMLVideoElement properties in the crash report I can extrapolate the following findings/remarks:

8/10 of the exceptions happened at the start of the video when the video.currentTime is 0.
When the video.currentTime is 0, at least one or both of video.mozPaintedFrames or video.mozPresentedFrames is 0. And video.mozParsedFrames and video.mozDecodedFrames are always higher than 0.
It seems like 2/10 of the exceptions happened midway when the video.currentTime is not 0.
When video.currentTime is 0 then video.paused is false. But when video.currentTime is higher than 0 then video.paused is true.
The video.networkState is always 2 (NETWORK_LOADING)
video.videoWidth and video.videoHeight are always set to a value.
Regarding video codecs I've seen both the mimetypes video/mp4 (1 time) and video/webm (5 times).
The video/webm mimetype has always the vp9 codec.
The video/mp4 mimetype could have been one of the following codecs (it was not captured exactly):

avc1.42001E, mp4a.40.2
avc1.64001F, mp4a.40.2
avc1.4d401f
avc1.4d4014
avc1.4d401e
avc1.4d400c
avc1.4d400b

Expected results:

The video should be drawable when the video is playing.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core

Moving to gfx as my understanding is that this is an issue of gfx reading back for the canvas.

Component: Audio/Video: Playback → Graphics

Hello Sotaro, from the description, do you agree it could be a problem on our side?

(Marking platform as Windows 10 as per bug title, note that the user agent mentions WebKit/Safari, not sure how that works)

OS: Unspecified → Windows 10
Priority: -- → P3
Hardware: Unspecified → Desktop

Resetting severity to default of --.

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is P3 (Backlog,) indicating it has been triaged, the bug's Severity is being updated to S3 (normal.)

Severity: normal → S3

@bpeers
The Chrome User Agent in my original comment is from the browser I've submitted this bugreport from. You can ignore it.

I wanted to let you know that I'm still daily catching multiple errors from my users on Firefox 83 in combination with Windows 10. The User Agent of those users is the last few weeks:
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0

I still have not received any reports from users on Linux or Mac, only from Windows 10. So we can be fairly certain that it's only a bug on Windows 10.

Did you expect Sotaro to provide feedback as mentioned in comment 3? There has been no reponse after 8 months.


@sotaro
Since it's been a while I can provide some additional information that might help narrowing down the cause.

As reported in the original comment the networkState is still always 2 (NETWORK_LOADING). Additional information that might be helpful is that the readyState is always 4 (HAVE_ENOUGH_DATA).

Also, the video element in the canvasContext.drawImage(videoElement, 0, 0) call is the video element on the YouTube.com/watch webpage. It can be retrieved via document.querySelector('.html5-main-video') selector.


@jimm
Please let me know if you need additional information. I'm willing to provide the information you need.

Flags: needinfo?(jmathies)

(In reply to Bert Peers [:bpeers] from comment #3)

Hello Sotaro, from the description, do you agree it could be a problem on our side?

(Marking platform as Windows 10 as per bug title, note that the user agent mentions WebKit/Safari, not sure how that works)

Sorry, I did not notice it, since needinfo was not set. I am going to look into it.

Flags: needinfo?(sotaro.ikeda.g)

(In reply to Wessel Kroos from comment #0)

When calling canvasContext.drawImage(video, ..) a NS_ERROR_NOT_AVAILABLE is thrown.

It seems that "NS_ERROR_NOT_AVAILABLE" was replaced by "Passed-in image is "broken"" since Firefox 85 by Bug 1681418.

(In reply to Wessel Kroos from comment #6)

I still have not received any reports from users on Linux or Mac, only from Windows 10. So we can be fairly certain that it's only a bug on Windows 10.

On Windows, gpu process exists, it might affect to it.

Since Firefox 84, the place of video decoding was moved from GPU process to Rdd process by Bug 1595994. It might affect to the symptom.

jya knows well about video element.

: jya, can you comment to this bug?

Flags: needinfo?(sotaro.ikeda.g) → needinfo?(jyavenard)

(In reply to Sotaro Ikeda [:sotaro] from comment #11)

Since Firefox 84, the place of video decoding was moved from GPU process to Rdd process by Bug 1595994. It might affect to the symptom.

not for windows H264 hardware decoder.

I need an example to reproduce the issue myself.

Note that I won't be available to help on this from January 4th.

Flags: needinfo?(jyavenard)
Regressed by: 1681418

I don't see how this bug is a regression from bug 1681418? That just changed the error message by a more helpful one.

Yea, bug 1681418 seems not related to this bug.

No longer regressed by: 1681418

:sotaro I can confirm that the crash report contains the new error message. I've received the InvalidStateError since Firefox 85.

Here is the crash report information from a Firefox 86 user that I've received 2 days ago:

InvalidStateError: CanvasRenderingContext2D.drawImage: Passed-in image is "broken"
Windows Version: 10
Firefox Version: 86.0
The drawImage source:

HTMLVideoElement
{
  "allowedToPlay": true,
  "autoplay": false,
  "buffered": {},
  "controls": false,
  "crossOrigin": null,
  "currentSrc": "blob:https://www.youtube.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "currentTime": 0,
  "defaultMuted": false,
  "defaultPlaybackRate": 1,
  "duration": 301.061,
  "ended": false,
  "error": null,
  "HAVE_CURRENT_DATA": 2,
  "HAVE_ENOUGH_DATA": 4,
  "HAVE_FUTURE_DATA": 3,
  "HAVE_METADATA": 1,
  "HAVE_NOTHING": 0,
  "height": 0,
  "loop": false,
  "mediaKeys": null,
  "mozAudioCaptured": false,
  "mozDecodedFrames": 1,
  "mozFragmentEnd": 301.061,
  "mozFrameDelay": 0,
  "mozHasAudio": true,
  "mozPaintedFrames": 0,
  "mozParsedFrames": 1,
  "mozPresentedFrames": 0,
  "mozPreservesPitch": true,
  "muted": false,
  "NETWORK_EMPTY": 0,
  "NETWORK_IDLE": 1,
  "NETWORK_LOADING": 2,
  "NETWORK_NO_SOURCE": 3,
  "networkState": 2,
  "paused": true,
  "playbackRate": 1,
  "played": {},
  "poster": "",
  "preload": "",
  "readyState": 4,
  "seekable": {},
  "seeking": false,
  "src": "blob:https://www.youtube.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "srcObject": null,
  "textTracks": {},
  "videoHeight": 144,
  "videoWidth": 256,
  "volume": 1,
  "width": 0
}
Flags: needinfo?(jmathies) → needinfo?(sotaro.ikeda.g)

(In reply to Wessel Kroos from comment #16)

InvalidStateError: CanvasRenderingContext2D.drawImage: Passed-in image is "broken"

The error seemed to be detected at the following,.

(In reply to Wessel Kroos from comment #16)

:sotaro I can confirm that the crash report contains the new error message. I've received the InvalidStateError since Firefox 85.

The new error message was added by Bug 1681418.

See Also: → 1681418

As in Comment 9, function calls in nsLayoutUtils::SurfaceFromElement() might be failed.

(In reply to Wessel Kroos from comment #0)

8/10 of the exceptions happened at the start of the video when the video.currentTime is 0.
When the video.currentTime is 0, at least one or both of video.mozPaintedFrames or video.mozPresentedFrames is 0. And video.mozParsedFrames and video.mozDecodedFrames are always higher than 0.

"mozPresentedFrames is 0" mean that VideoSink did not do rendering.

nsLayoutUtils::SurfaceFromElement() seems to expect the followings.

-[1] When ready state of MediaElement is HAVE_NOTHING or HAVE_METADATA, it does not do rendering without exception.
-[2] When HTMLMediaElement::GetCurrentImage() does not return a valid image and if the ready state is not [1], CanvasRenderingContext2D::DrawImage() throws InvalidStateError.

Flags: needinfo?(sotaro.ikeda.g)
Flags: needinfo?(sotaro.ikeda.g)

:emilio, does HTMLMediaElement::GetCurrentImage() need to return a valid video frame image if ready state of the MediaElement is not HAVE_NOTHING nor HAVE_METADATA(HAVE_CURRENT_DATA, HAVE_ENOUGH_DATA, HAVE_FUTURE_DATA) for nsLayoutUtils::SurfaceFromElement()?

Flags: needinfo?(emilio)

I don't think so, but per spec we shouldn't throw unless we're an <img> <svg:image>`.

Flags: needinfo?(emilio)
Assignee: nobody → emilio
Flags: needinfo?(sotaro.ikeda.g)
Pushed by ealvarez@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d9b7bed96428
CanvasRenderingContext2D.drawImage shouldn't throw for e.g <video> if there's no valid surface to draw. r=sotaro
Flags: needinfo?(emilio)
Pushed by ealvarez@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ab82c7e299f4
CanvasRenderingContext2D.drawImage shouldn't throw for e.g <video> if there's no valid surface to draw. r=sotaro

Third time's the charm.

Flags: needinfo?(emilio)
Pushed by ealvarez@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/383fec2e815c
CanvasRenderingContext2D.drawImage shouldn't throw for e.g <video> if there's no valid surface to draw. r=sotaro
Pushed by emilio@crisal.io:
https://hg.mozilla.org/integration/autoland/rev/b0eba102423a
Allow timeout in RTCPeerConnection test now that we don't throw and insta-fail on android.
Pushed by malexandru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/12ee10496a34
Allow timeout in RTCRtpSender-replaceTrack a=wpt-fix
Backout by malexandru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e6ea3d7cc64d
Backed out 3 changesets for causing failures in test_eme_canvas_blocked.html

There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:emilio, could you have a look please?
For more information, please visit auto_nag documentation.

Flags: needinfo?(sotaro.ikeda.g)
Flags: needinfo?(emilio)

Yes, I need to green up some android tests that rely on our current behavior.

Flags: needinfo?(sotaro.ikeda.g)
Pushed by ealvarez@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4a58aec917b0
CanvasRenderingContext2D.drawImage shouldn't throw for e.g <video> if there's no valid surface to draw. r=sotaro
Status: UNCONFIRMED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → 90 Branch
Flags: needinfo?(emilio)
You need to log in before you can comment on or make changes to this bug.