1590030 - ImageBitmapRenderingContext.transferFromImageBitmap(imageBitmap) does not display ImageBitmap at <canvas> or <video> does not playback MediaStream from canvas.captureStream()

Status: NEW

(Core :: Graphics: Canvas2D, defect, P3)

Description

[guest271314]

Attachment: bitmaprenderer.html

User Agent: Mozilla/5.0 (X11; Linux i686; rv:71.0) Gecko/20100101 Firefox/71.0

Steps to reproduce:

1. Create a ImageBitmap with ImageData set to random pixel colors as source
2. Execute captureStream() on <canvas>
3. Attach mute and unmute events to MediaStreamTrack, loadedmetadata, canplay, play events to <video>
4. Execute transferFromImageBitmap(imageBitmap), requestFrame() on CanvasCaptureMediaStream

Actual results:

Nightly 71

1. The <canvas> is tranparent
2. The <video> is transparent
3. MediaStreamTrack unmute and mute events are not fired, <video> loadedmetadata canplay, play events are not fired

Firefox 69

1. The <canvas> displays the transferred ImageBitmap
2. The <video> is transparent
3. MediaStreamTrack unmute and mute events are not fired, <video> loadedmetadata canplay, play events are not fired

Chromium 79

1. The <canvas> displays the transferred ImageBitmap
2. The <video> displays the rendered frame from the <canvas>
3. MediaStreamTrack unmute is not fired, <video> loadedmetadata canplay, play events are fired, MediaStreamTrack mute event is fired

Expected results:

1. The <canvas> displays the transferred ImageBitmap
2. The <video> displays the rendered frame from the <canvas>
3. MediaStreamTrack unmute is fired, <video> loadedmetadata canplay, play events are fired, MediaStreamTrack mute event is fired

Comment 1

[Daniel Bodea [:danibodea]]

I have attempted to find the regressor of this issue, but these are my results:

• Nightly v72.0a1:
  the image would show (on the left), but the video is white (on the right)
  the video would not autoplay
  the volume controls can be used.
• Beta v71.0b3:
  the image would show (on the left), but the video is white (on the right)
  the video would not autoplay
  the volume controls can be used.
• Release v70.0:
  the image would show (on the left), but the video is white (on the right)
  the video would not autoplay
  the volume controls can be used.
• ESR68 v68.2.0esr:
  the image would show (on the left), but the video is white (on the right)
  the video WOULD autoplay
  the volume controls can be used.
• ESR60v60.8.0esr:
  the image would show (on the left), but the video is white (on the right)
  the video WOULD autoplay
  the volume controls can be used.

Furthermore, I looked a few versions back using mozregression to find a build that does not reproduce the issue:

• Nightly v55.0a1:
  the image would show (on the left), but the video is white (on the right)
  the video WOULD autoplay
  the volume controls can be used.
• Nightly v50.0a1:
  Test results on this version seem to be inconclusive because the content of the test page would not even render (white page).
• Nightly v45.0a1:
  Test results on this version seem to be inconclusive because the content of the test page would not even render (white page).
• Chrome v77.0.3865.120:
  the image would show (on the left) and the video will also display the same image thumbnail (on the right)
  the video would not autoplay
  the volume controls can NOT be used.

In conclusion, I think we can safely say that this is not a recent regression and the behavior seen in Chrome cannot be found in old versions of Firefox. If another test page cand be provided (one that can be used on older versions of Firefox), the regression could be attempted again.
Until then, I will assume that this issue is not a regression. Please NI me if I can help further. Thank you.

Comment 2

[guest271314]

(In reply to Bodea Daniel [:danibodea] from comment #1)

I have attempted to find the regressor of this issue, but these are my results:

• Nightly v72.0a1:
  the image would show (on the left), but the video is white (on the right)
  the video would not autoplay
  the volume controls can be used.
• Beta v71.0b3:
  the image would show (on the left), but the video is white (on the right)
  the video would not autoplay
  the volume controls can be used.
• Release v70.0:
  the image would show (on the left), but the video is white (on the right)
  the video would not autoplay
  the volume controls can be used.
• ESR68 v68.2.0esr:
  the image would show (on the left), but the video is white (on the right)
  the video WOULD autoplay
  the volume controls can be used.
• ESR60v60.8.0esr:
  the image would show (on the left), but the video is white (on the right)
  the video WOULD autoplay
  the volume controls can be used.

Furthermore, I looked a few versions back using mozregression to find a build that does not reproduce the issue:

• Nightly v55.0a1:
  the image would show (on the left), but the video is white (on the right)
  the video WOULD autoplay
  the volume controls can be used.
• Nightly v50.0a1:
  Test results on this version seem to be inconclusive because the content of the test page would not even render (white page).
• Nightly v45.0a1:
  Test results on this version seem to be inconclusive because the content of the test page would not even render (white page).
• Chrome v77.0.3865.120:
  the image would show (on the left) and the video will also display the same image thumbnail (on the right)
  the video would not autoplay
  the volume controls can NOT be used.

In conclusion, I think we can safely say that this is not a recent regression and the behavior seen in Chrome cannot be found in old versions of Firefox. If another test page cand be provided (one that can be used on older versions of Firefox), the regression could be attempted again.
Until then, I will assume that this issue is not a regression. Please NI me if I can help further. Thank you.

Re autoplay: At Chromium 79 the video will autoplay when launched with the flag autoplay-policy=no-user-gesture-required.

volume controls: There is no audio track.

Firefox and Nightly do draw the image when "2d" is passed to getContext(). However, "2d" is not implemented for OffscreenCanvas at Mozilla browsers.

Comment 3

[guest271314]

Is transferFromImageBitmap no longer implemented?

Comment 4

[guest271314]

At Nightly 73 transferFromImageBitmap() does diplay ImageBitmap at <canvas> and playback of MediaStream from captureStream() does proceed.