1177793 - black screen when using a mozCaptureStream

Status: VERIFIED FIXED

(Core :: Audio/Video: Playback, defect, P3)

Description

[Paul Rouget [:paul]]

This script:

let v = document.querySelector('video');
let stream = v.mozCaptureStream();
let streamURL = URL.createObjectURL(stream);
let div = document.createElement('div');
document.body.appendChild(div);
div.outerHTML = `
<div style="position:absolute;top:0;right:0;border:1px solid red">
<video controls autoplay>
<source src="${streamURL}"></source>
</video>
</div>
`;

… duplicates a video stream.

Running with this page works for example:

http://people.mozilla.org/~prouget/demos/simpleVideo/video.html

But it doesn't work on youtube.com and most of the videos online. The video pauses, and the new video doesn't load.

I'm wondering if it could be a CSP/Cors issue.

Comment 1

[Paul Rouget [:paul]]

Roc, if I'm not mistaken, you worked on bug 792675, so maybe you know what's going on.

Comment 2

[Paul Rouget [:paul]]

With a recent nightly build, I get these assertions:

###!!! ASSERTION: Wrong type: 'aSource->GetType() == C::StaticType()', file /Users/paul/mozilla/src/media/webrtc/signaling/../../../dom/media/MediaSegment.h, line 152
###!!! ASSERTION: Track with this ID already exists: '!FindTrack(aID)', file /Users/paul/mozilla/src/dom/media/StreamBuffer.h, line 213

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

Those assertions smell like something Andreas has been wrestling with.

Comment 4

[Andreas Pehrson [:pehrsons]]

CORS and mozCaptureStream used to stop playback of the original media element until bug 1174077 landed quite recently. Now we, instead of blocking output to the stream, keep feeding the stream silence in the CORS case. Instead of a complete pause, video (and time) should still be going after the stream gets blocked due to CORS.

The asserts look bad though. Something must be going bananas!

Comment 5

[Paul Rouget [:paul]]

Some context: we are trying to implement a Picture In Picture feature for web pages. We use mozCaptureStream.

I think there are 2 issues here:

1) calling mozCaptureStream() freezes some videos (youtube for example)

I don't know why.

2) using the stream doesn't work because of a principal/cors issue

For 2, this is what's going on:

IsSameOrigin is false here: http://hg.mozilla.org/mozilla-central/file/eaf4f9b45117/dom/media/MediaDecoderStateMachine.cpp#l550
Set here: http://hg.mozilla.org/mozilla-central/file/eaf4f9b45117/dom/html/HTMLMediaElement.cpp#l3919

Which makes sense (or does it? Is it normal that we can play a video but not the stream in the same document?).

Anyway, in my case, the `mediastream:…` URL is loaded in a `chrome://…` document (I built a protocol handler that opens a `mediastream:…` URL in a tab within a video tag), where the principal is the system principal, and `mozCaptureStream` is also called inside chrome code.

So both the stream and the document use system principal.

The second stream (the one from my chrome page) should allow rendering of the video in a the chrome document by calling updateHTMLMediaElement::NotifyDecoderPrincipalChanged() but it's not happening.

Maybe in MediaDecoderStateMachine::SendStreamData we should check the principal of the stream, and if it's a system principal, we should set IsSameOrigin to true. 

But the DOMMediaStream (which holds the principal) is not reachable.

Hope that makes sense. Any idea what is the right approach here?

Comment 6

[Andreas Pehrson [:pehrsons]]

I also see it pause in Nightly. It really shouldn't with bug 1174077. Just to throw it out there: could it be because youtube is playing a mediasource rather than a file?

Comment 7

[Paul Rouget [:paul]]

> could it be because youtube is playing a mediasource rather than a file?

It might be it. I can only reproduce on youtube, and youtube is the only place I know that use a mediastream url.

Comment 8

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Attachment: testcase

When I run this as a local file on a reasonably recent Nightly build, playback works but I get black in the MediaStream output. We recently changed that behavior --- it used to just block --- so I guess you're using an older build.

Both behaviors happen because mozCaptureStream() stops cross-origin media data from entering MediaStreams. This got added in bug 856361 to block cross-origin audio from entering the MediaStreamGraph system where it could eventually be examined by Web Audio ScriptProcessorNodes. But I think it could be removed since we have infrastructure to track origins through MediaStreams and MediaStreamAudioSourceNode does its own origin checks.

Comment 9

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Oops, I should have refreshed the bug before I started looking into it.

Comment 10

[Robert O'Callahan (:roc) (email my personal email if necessary)]

It seems to me from looking at the code that cross-origin video *should* be considered same-origin if the principle of the <video> element's document is system principal. Paul, maybe you could add some logging to HTMLMediaElement::IsCORSSameOrigin to dump the principals when that check returns false?

No idea why stream capture wouldn't work with MSE. MSE doesn't work at all for me on Linux, apparently. Querying jya, who might be able to help.

Comment 11

[Paul Rouget [:paul]]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #10)
> It seems to me from looking at the code that cross-origin video *should* be
> considered same-origin if the principle of the <video> element's document is
> system principal. Paul, maybe you could add some logging to
> HTMLMediaElement::IsCORSSameOrigin to dump the principals when that check
> returns false?

IsSameOrigin is check in the decoder. Value is set via MediaDecoder::UpdateSameOriginStatus(), which is supposed to be called from HTMLMediaElement::NotifyDecoderPrincipalChanged(). But this is not called.

More precisely, this is what I do:

0. play video in a tab. HTMLMediaElement::NotifyDecoderPrincipalChanged() is called.
1. get mediastream: url from chrome code
2. open a new window (chrome://)
3. this new window build a <video> tag and set the mediastream url. HTMLMediaElement::NotifyDecoderPrincipalChanged()is not called.

Comment 12

[Paul Rouget [:paul]]

But I don't think we want to update the principal at the decoder level. Wouldn't that allow bad things to happen in term of security?

The principal ** of the stream ** should be checked in the state machine code.

I don't know much about security and our media code, so please correct me if I'm wrong.

Comment 13

[Andreas Pehrson [:pehrsons]]

To my understanding the principal should be checked by each consumer of the stream. Producers should keep the principal updated but always feed the stream real data. That might not currently be the case though, for instance the MediaDecoderStateMachine feeds the stream silence/black frames if it's streaming a cross-origin resource.

Clearing ni? here until we have some more info on MSE.

Comment 14

[Paul Rouget [:paul]]

Paul, are you still looking at the permission issue?

Comment 15

[Paul Adenot (:padenot)]

My case turned out to be simpler that what is needed here, so I guess we still have work to do.

Have you managed to repro the permission issue with non-MSE videos ?

Comment 16

[Paul Rouget [:paul]]

Comment 5 describes 2 bugs.

Let's keep this bug for (2), which is not specific to MSE. It happens with any video.

Let's talk about (1) in a different bug: bug 1181981

Comment 17

[Paul Rouget [:paul]]

Paul, the permission issue happens with any video with a different origin than the page embedding it.

Comment 18

[Paul Rouget [:paul]]

(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #13)
> To my understanding the principal should be checked by each consumer of the
> stream.

Where is the consumer code?
At the consumer level, do we have access to the DOMMediaStream?

Comment 19

[Andreas Pehrson [:pehrsons]]

(In reply to Paul Rouget [:paul] from comment #18)
> (In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #13)
> > To my understanding the principal should be checked by each consumer of the
> > stream.
> 
> Where is the consumer code?
> At the consumer level, do we have access to the DOMMediaStream?

Sure. Here's a good starting point: https://dxr.mozilla.org/mozilla-central/search?q=%2Bcallers%3Amozilla%3A%3ADOMMediaStream%3A%3AGetPrincipal%28%29

Comment 20

[Paul Rouget [:paul]]

Let's imagine these scenarios:

1)

In: http://bar.com/index.html

<video src="http://bar.com/video.webm">
<script>
  // build a mediastream URL: mediastream:abc
</script>

In: http://foo.com/index.html

<video src="mediastream:abc">


Should the video play?

2)

In: http://foo.com/index.html

<video src="http://bar.com/video.webm">
<script>
  // build a mediastream URL: mediastream:abc
</script>

In: http://foo.com/index.html

<video src="mediastream:abc">


Should the video play?

3)

In: http://bar.com/index.html

<video src="http://foo.com/video.webm">
<script>
  // build a mediastream URL: mediastream:abc
</script>

In: http://foo.com/index.html

<video src="mediastream:abc">


Should the video play?

Comment 21

[Mantaroh Yoshinaga[:mantaroh]]

bug 1180589 - Creating the TV Simulator.
This bug used the mozCaptureStream in order to provide MediaStream loading local video file.

I tried simple code on b2g desktop 2.2 and mozillacentral.
--------------------------------------------------------
  var videoElem = document.createElement("video");
  videoElem.autoplay = true;
  videoElem.src = "./data/sample.ogg";
  
  var stream = videoElem.mozCaptureStream();
  var video1 = document.getElementById("video1");
  video1.mozSrcObject = stream;
--------------------------------------------------------
B2G Desktop ver 2.2 : work well.
B2G Desktop m-c     : not work.

Perhaps this bug block bug 1180589.(see bug 1180589 comment #58)

Comment 22

[Jean-Yves Avenard [:jya]]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #10)

> No idea why stream capture wouldn't work with MSE. MSE doesn't work at all
> for me on Linux, apparently. Querying jya, who might be able to help.

MSE is now fully supported on Linux (both webm and h264 (if FFmpeg's libavcodec is installed).

Not sure how I could help further here.

Sorry for the delay in answering (if 6 months can be considered a "delay")

Comment 23

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Mass change P2 -> P3

Comment 24

[Maarten Breddels]

I believe this is the bug report (at least nr 2 from https://bugzilla.mozilla.org/show_bug.cgi?id=1177793#c5) why I'm getting a black screen. I basically want:
==============
var video1 = document.getElementById("video1");
video1.autoplay = true;
video1.src = "https://webrtc.github.io/samples/src/video/chrome.mp4";

... after it started playing ...
  
var stream = video1.mozCaptureStream();
var video2 = document.getElementById("video2");
video2.autoplay = true;
video2.srcObject = stream;
==============

I believe the issue here is that IsSameOrigin is false right? I cannot find this back in https://hg.mozilla.org/mozilla-central/file/tip/dom/media/MediaDecoderStateMachine.cpp but I can still confirm I get a black screen for mozilla 55.

Is there an issue why this cannot be fixed, or is there really a security issue here?

Regards,

Maarten

Comment 25

[Randell Jesup [:jesup] (needinfo me)]

jib, thoughts?  (baku is on vacation; he'd also be good to look)

Comment 26

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

(In reply to Maarten Breddels from comment #24)
> is there really a security issue hereCT

Note that comment 2 is about allowing cross-origin streams in privileged (chrome) code AFAICT.
If yours is a regular web page, then you're not allowed the bits of that video [1]. The spec [2] says:

    "Media elements can render media resources from origins that differ from the origin of the media element. In those cases, the contents of the resulting MediaStreamTrack MUST be protected from access by the document origin."

It goes on to explain how protection measures differ. Try https://jsfiddle.net/jib1/1kz9hfaL/ to see a couple.

Technically, since the only thing your brief example does with the stream is to play it, there's no issue, but it's also rather pointless, so I assume it's a stand-in for other stream sinks?

Unlike canvas, we don't support "origin-dirty" streams, we just blank them instead. I suppose we could add support for that, and instead blank at the point of contact with sinks like MediaRecorder, RTCPeerConnection and AudioContext.createMediaStreamSource(), but the use-cases seem limited.

[1] https://en.wikipedia.org/wiki/Same-origin_policy
[2] https://w3c.github.io/mediacapture-fromelement/#security-considerations

Comment 27

[Maarten Breddels]

Dear Jan-Ivar,

thanks for the explanation, I did not consider/understand the full security issue, but I think I do now. Basically, you don't want to expose the video pixels that could come from private sources, but displaying them is ok.
I was developing this with google chrome, which does support this (although captureStream is still behind standard disabled flags, chrome 60).
To give you some background, it's part of a library [1] with a model/view concept where the mediastream is the model (implemented for video with an invisible video tag), and you can display it, which is implemented again using a video tag (this time visible). This could be circumvented drawing it to a canvas, but my guess is that would waste cpu cycles. Although it would be of more limited use, since this video would not be possible to share over webrtc, it could be used to draw in a WebGL scene for instance [2,3].

Regards,

Maarten Breddels

[1] https://ipywebrtc.readthedocs.io/en/latest/
[2] https://ipyvolume.readthedocs.io/en/latest/
[3] https://twitter.com/maartenbreddels/status/894983501996584961

Comment 28

[Daniel Veditz [:dveditz]]

Hiding this bug for now while we decide how to handle this. Need to tell the Chrome security team. After that could perhaps just hide the last few comments and un-hide this bug.

Comment 29

[Maarten Breddels]

Actually, the video->video is not a security bug (since still you cannot get the data). I did notice however, I can send a cross-origin stream over WebRTC, which must be a bug. Feel free to submit the Google Chrome team, or let me know where to do that.

Comment 30

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

(In reply to Maarten Breddels from comment #27)

Wow, that looks awesome, thanks for sharing! Also, thanks for the info on Chrome. I've updated the fiddle to work there: https://jsfiddle.net/jib1/1kz9hfaL/ - Unfortunately, that looks like a security bug in Chrome.

> This could be circumvented drawing it to a canvas

Actually no, because as the fiddle shows, even in Chrome you'll get:

  canvas getImageData(): SecurityError: Failed to execute 'getImageData' on 'CanvasRenderingContext2D': The canvas
  has been tainted by cross-origin data.
  canvas captureStream(): SecurityError: Failed to execute 'captureStream' on 'HTMLCanvasElement': Canvas is not origin-clean.

But switch s/video1/video2/ here:

   canvas3.getContext('2d').drawImage(video2, 0, 0, 160, 90);

.. and I see two videos and getImageData() now returns [Object ImageData] in Chrome Canary (62), without any chrome flags AFAICT.

Dan will email the Chrome team.

Comment 31

[Maarten Breddels]

Both Firefox and Chrome can do:
  canvas3.getContext('2d').drawImage(video1, 0, 0, 160, 90);
So you can mimic video playback from a tainted source to a canvas (by doing a timer etc), but Chrome can do this directly to a HTMLVideoElement, while Firefox cannot (but Chrome does not allow you can do getImageData, which means it is safe). Meaning a workaround to get similar behaviour on Firefox would be this canvas+timer, which isn't very elegant.

Comment 32

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Maarten, yes I see what you mean now about simulating playback.

You're also right about the real Chrome sec bug appearing to be webrtc: https://jsfiddle.net/jib1/1kz9hfaL/9/

Comment 33

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

In light of the use-case in comment 27 we should perhaps consider supporting tainted MediaStreamTracks instead of blanking them.

Comment 34

[Daniel Veditz [:dveditz]]

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #30)
> (In reply to Maarten Breddels from comment #27)
> 
> But switch s/video1/video2/ here:
> 
>    canvas3.getContext('2d').drawImage(video2, 0, 0, 160, 90);
> 
> .. and I see two videos and getImageData() now returns [Object ImageData] in
> Chrome Canary (62), without any chrome flags AFAICT.

But you don't get any image on the canvas, so it makes sense because it's not tainted yet.

> Dan will email the Chrome team.

It seems a broken or incomplete that you can drawImage() from a regular video but not from a stream captured from a video, but that seems to be what's happening in chrome:
https://jsfiddle.net/dveditz/2t673qov/

Doesn't look like a security bug to me. Letting the second video play and then blocking export seems more useful than not letting the cross-origin play at all. Even allowing the second drawImage() would be OK as long as we propagate the taint correctly. It's certainly reasonable to draw a line somewhere and decide going further is not worth the work, but it would be nice if we and Chrome drew the line in similar places :-)

I haven't looked at your webrtc case yet.

Comment 35

[Martin Thomson [:mt:]]

Yeah, looking at the fiddle, the behaviour of the canvas1 is correct.

The capture of video1 to video2 should work though.  That stream should be cross-origin tainted (as discussed).  We taint MediaStreamTrack coming out of WebRTC (if it has a peerIdentity and isolation is requested), so we know that it works.

What should happen in Dan's fiddle is that you get two playing videos at the top.  A screen grab in the middle (of the same frame ideally), and a video that shows that single frame in the bottom.

The media should send black over WebRTC, but we won't find out until capture taints rather than captures black.

Comment 36

[Maarten Breddels]

> It seems a broken or incomplete that you can drawImage() from a regular video but not from a stream captured from a video, but > that seems to be what's happening in chrome:
> https://jsfiddle.net/dveditz/2t673qov/

You can only drawImage/captureStream after onplayread, video1 was already playing, so that worked, see this fix:
https://jsfiddle.net/j5yxec3b/

Yeah, I think capture from a tainted source and rendering that on a 'save' element (like a video element marked as tainted) would be behaviour as expected (since it doesn't do any harm).

I was also wondering about WebGL, and there Google Chrome has another security issue, you can render a video to a WebGL and read out that canvas. I couldn't set up a small fiddle, but I can show you here:
https://www.astro.rug.nl/~breddels/ipyvolume/tainted/
Note that this also went the video->captureStream->video path, directly passing the original video element to WebGL behaves save.

Comment 37

[Daniel Veditz [:dveditz]]

The Chrome issue is https://bugs.chromium.org/p/chromium/issues/detail?id=756877

Comment 38

[Andreas Pehrson [:pehrsons]]

As Martin mentions in comment 35 we have support for tainted tracks. The blanking [1] predates much of the logic around tainted tracks though.

Removing the blanking shouldn't be much further away than a mochitest or two.

On the other hand, it doesn't look like principal changes for MediaDecoder propagate properly to DecodedStream, thus to individual media chunks, per [2], [3]. That will have to be resolved first for un-tainting a track (it'll still get tainted on a principal change at [4]).


[1] http://searchfox.org/mozilla-central/rev/b258e6864ee3e809d40982bc5d0d5aff66a20780/dom/media/mediasink/DecodedStream.cpp#637
[2] http://searchfox.org/mozilla-central/rev/b258e6864ee3e809d40982bc5d0d5aff66a20780/dom/media/MediaDecoder.cpp#932
[3] http://searchfox.org/mozilla-central/rev/b258e6864ee3e809d40982bc5d0d5aff66a20780/dom/media/MediaDecoderStateMachine.cpp#2800
[4] http://searchfox.org/mozilla-central/rev/b258e6864ee3e809d40982bc5d0d5aff66a20780/dom/html/HTMLMediaElement.cpp#3234

Comment 39

[Daniel Veditz [:dveditz]]

(In reply to Daniel Veditz [:dveditz] from comment #37)
> The Chrome issue is
> https://bugs.chromium.org/p/chromium/issues/detail?id=756877

They used that bug to fix a null-deref crash I found while trying to create an .html file for the testcase. The actual tainting issue was spun off into https://bugs.chromium.org/p/chromium/issues/detail?id=761622

Comment 40

[Maarten Breddels]

I cannot see those chromium issues, can I get access to them, and will that issue in this one be public once fixed?

Comment 41

[Daniel Veditz [:dveditz]]

I'll unhide this when Google unhides their bug.

Do you have an account on their bug tracker? If so I can ask that they add you to it.

Comment 42

[Maarten Breddels]

Thanks, that would be maartenbreddels _ gmail.com

Comment 43

[Daniel Veditz [:dveditz]]

The Chrome bugs were fixed in Chrome 62... we can unhide this now. Since we were correctly propagating the cross-origin taint and now Chrome's behavior matches we're not going to "fix" this as requested

Comment 44

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

I think the request here was still valid. I believe we still output black rather than tainted video in the face of tainted input. We should try to match what Chrome does here for local playback, which was being discussed here I think. Reopening for consideration.

Came across this while trying to figure out why https://webrtc.github.io/samples/src/content/capture/video-pc/ emits black in Firefox but not Chrome.

Comment 45

[Andreas Pehrson [:pehrsons]]

I updated your old fiddle for a manual test case.

Comment 46

[Andreas Pehrson [:pehrsons]]

I have a patch for this but it needs tests.

Comment 47

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1177793 - Pass principal instead of forcing black when capturing cross-origin media resource. r?jya!,jib!

Comment 48

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1177793 - Modernize test_streams_capture_origin.html. r?jib

Depends on D36896

Comment 49

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1177793 - Update mochitest. r?jib

Depends on D36917

Comment 50

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/18a62a05982e
Pass principal instead of forcing black when capturing cross-origin media resource. r=jya,jib
https://hg.mozilla.org/integration/autoland/rev/46ad935ecd17
Modernize test_streams_capture_origin.html. r=jib
https://hg.mozilla.org/integration/autoland/rev/f775de3b67f6
Update mochitest. r=jib

Comment 51

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/18a62a05982e
https://hg.mozilla.org/mozilla-central/rev/46ad935ecd17
https://hg.mozilla.org/mozilla-central/rev/f775de3b67f6

Comment 52

[Bogdan Maris, Desktop QA]

Reproduced the issue in an old Nightly build before the fix. Verified that no black screen appears using the fiddle from comment 45 in 70.0b9 across platforms (macOS 10.13, Windows 10 64bit and Ubuntu 18.04).