Closed Bug 1177793 Opened 4 years ago Closed 2 months ago

black screen when using a mozCaptureStream

Categories

(Core :: Audio/Video: Playback, defect, P3)

defect

Tracking

()

RESOLVED FIXED
mozilla70
Tracking Status
firefox70 --- fixed

People

(Reporter: paul, Assigned: pehrsons)

References

Details

(Keywords: sec-other, Whiteboard: Keep hidden until Chrome bug is fixed)

Attachments

(4 files)

This script:

let v = document.querySelector('video');
let stream = v.mozCaptureStream();
let streamURL = URL.createObjectURL(stream);
let div = document.createElement('div');
document.body.appendChild(div);
div.outerHTML = `
<div style="position:absolute;top:0;right:0;border:1px solid red">
<video controls autoplay>
<source src="${streamURL}"></source>
</video>
</div>
`;

… duplicates a video stream.

Running with this page works for example:

http://people.mozilla.org/~prouget/demos/simpleVideo/video.html

But it doesn't work on youtube.com and most of the videos online. The video pauses, and the new video doesn't load.

I'm wondering if it could be a CSP/Cors issue.
Roc, if I'm not mistaken, you worked on bug 792675, so maybe you know what's going on.
Flags: needinfo?(roc)
With a recent nightly build, I get these assertions:

###!!! ASSERTION: Wrong type: 'aSource->GetType() == C::StaticType()', file /Users/paul/mozilla/src/media/webrtc/signaling/../../../dom/media/MediaSegment.h, line 152
###!!! ASSERTION: Track with this ID already exists: '!FindTrack(aID)', file /Users/paul/mozilla/src/dom/media/StreamBuffer.h, line 213
Those assertions smell like something Andreas has been wrestling with.
Flags: needinfo?(pehrsons)
CORS and mozCaptureStream used to stop playback of the original media element until bug 1174077 landed quite recently. Now we, instead of blocking output to the stream, keep feeding the stream silence in the CORS case. Instead of a complete pause, video (and time) should still be going after the stream gets blocked due to CORS.

The asserts look bad though. Something must be going bananas!
Some context: we are trying to implement a Picture In Picture feature for web pages. We use mozCaptureStream.

I think there are 2 issues here:

1) calling mozCaptureStream() freezes some videos (youtube for example)

I don't know why.

2) using the stream doesn't work because of a principal/cors issue

For 2, this is what's going on:

IsSameOrigin is false here: http://hg.mozilla.org/mozilla-central/file/eaf4f9b45117/dom/media/MediaDecoderStateMachine.cpp#l550
Set here: http://hg.mozilla.org/mozilla-central/file/eaf4f9b45117/dom/html/HTMLMediaElement.cpp#l3919

Which makes sense (or does it? Is it normal that we can play a video but not the stream in the same document?).

Anyway, in my case, the `mediastream:…` URL is loaded in a `chrome://…` document (I built a protocol handler that opens a `mediastream:…` URL in a tab within a video tag), where the principal is the system principal, and `mozCaptureStream` is also called inside chrome code.

So both the stream and the document use system principal.

The second stream (the one from my chrome page) should allow rendering of the video in a the chrome document by calling updateHTMLMediaElement::NotifyDecoderPrincipalChanged() but it's not happening.

Maybe in MediaDecoderStateMachine::SendStreamData we should check the principal of the stream, and if it's a system principal, we should set IsSameOrigin to true. 

But the DOMMediaStream (which holds the principal) is not reachable.

Hope that makes sense. Any idea what is the right approach here?
I also see it pause in Nightly. It really shouldn't with bug 1174077. Just to throw it out there: could it be because youtube is playing a mediasource rather than a file?
> could it be because youtube is playing a mediasource rather than a file?

It might be it. I can only reproduce on youtube, and youtube is the only place I know that use a mediastream url.
Attached file testcase
When I run this as a local file on a reasonably recent Nightly build, playback works but I get black in the MediaStream output. We recently changed that behavior --- it used to just block --- so I guess you're using an older build.

Both behaviors happen because mozCaptureStream() stops cross-origin media data from entering MediaStreams. This got added in bug 856361 to block cross-origin audio from entering the MediaStreamGraph system where it could eventually be examined by Web Audio ScriptProcessorNodes. But I think it could be removed since we have infrastructure to track origins through MediaStreams and MediaStreamAudioSourceNode does its own origin checks.
Flags: needinfo?(roc)
Oops, I should have refreshed the bug before I started looking into it.
It seems to me from looking at the code that cross-origin video *should* be considered same-origin if the principle of the <video> element's document is system principal. Paul, maybe you could add some logging to HTMLMediaElement::IsCORSSameOrigin to dump the principals when that check returns false?

No idea why stream capture wouldn't work with MSE. MSE doesn't work at all for me on Linux, apparently. Querying jya, who might be able to help.
Flags: needinfo?(jyavenard)
(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #10)
> It seems to me from looking at the code that cross-origin video *should* be
> considered same-origin if the principle of the <video> element's document is
> system principal. Paul, maybe you could add some logging to
> HTMLMediaElement::IsCORSSameOrigin to dump the principals when that check
> returns false?

IsSameOrigin is check in the decoder. Value is set via MediaDecoder::UpdateSameOriginStatus(), which is supposed to be called from HTMLMediaElement::NotifyDecoderPrincipalChanged(). But this is not called.

More precisely, this is what I do:

0. play video in a tab. HTMLMediaElement::NotifyDecoderPrincipalChanged() is called.
1. get mediastream: url from chrome code
2. open a new window (chrome://)
3. this new window build a <video> tag and set the mediastream url. HTMLMediaElement::NotifyDecoderPrincipalChanged()is not called.
But I don't think we want to update the principal at the decoder level. Wouldn't that allow bad things to happen in term of security?

The principal ** of the stream ** should be checked in the state machine code.

I don't know much about security and our media code, so please correct me if I'm wrong.
To my understanding the principal should be checked by each consumer of the stream. Producers should keep the principal updated but always feed the stream real data. That might not currently be the case though, for instance the MediaDecoderStateMachine feeds the stream silence/black frames if it's streaming a cross-origin resource.

Clearing ni? here until we have some more info on MSE.
Flags: needinfo?(pehrsons)
Paul, are you still looking at the permission issue?
Flags: needinfo?(padenot)
My case turned out to be simpler that what is needed here, so I guess we still have work to do.

Have you managed to repro the permission issue with non-MSE videos ?
Flags: needinfo?(padenot)
Comment 5 describes 2 bugs.

Let's keep this bug for (2), which is not specific to MSE. It happens with any video.

Let's talk about (1) in a different bug: bug 1181981
Summary: mozGetMediaSource doesn't work on youtube → black screen when using a mozCaptureStream
Paul, the permission issue happens with any video with a different origin than the page embedding it.
(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #13)
> To my understanding the principal should be checked by each consumer of the
> stream.

Where is the consumer code?
At the consumer level, do we have access to the DOMMediaStream?
Flags: needinfo?(pehrsons)
(In reply to Paul Rouget [:paul] from comment #18)
> (In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #13)
> > To my understanding the principal should be checked by each consumer of the
> > stream.
> 
> Where is the consumer code?
> At the consumer level, do we have access to the DOMMediaStream?

Sure. Here's a good starting point: https://dxr.mozilla.org/mozilla-central/search?q=%2Bcallers%3Amozilla%3A%3ADOMMediaStream%3A%3AGetPrincipal%28%29
Flags: needinfo?(pehrsons)
Let's imagine these scenarios:

1)

In: http://bar.com/index.html

<video src="http://bar.com/video.webm">
<script>
  // build a mediastream URL: mediastream:abc
</script>

In: http://foo.com/index.html

<video src="mediastream:abc">


Should the video play?

2)

In: http://foo.com/index.html

<video src="http://bar.com/video.webm">
<script>
  // build a mediastream URL: mediastream:abc
</script>

In: http://foo.com/index.html

<video src="mediastream:abc">


Should the video play?

3)

In: http://bar.com/index.html

<video src="http://foo.com/video.webm">
<script>
  // build a mediastream URL: mediastream:abc
</script>

In: http://foo.com/index.html

<video src="mediastream:abc">


Should the video play?
Blocks: graphene
bug 1180589 - Creating the TV Simulator.
This bug used the mozCaptureStream in order to provide MediaStream loading local video file.

I tried simple code on b2g desktop 2.2 and mozillacentral.
--------------------------------------------------------
  var videoElem = document.createElement("video");
  videoElem.autoplay = true;
  videoElem.src = "./data/sample.ogg";
  
  var stream = videoElem.mozCaptureStream();
  var video1 = document.getElementById("video1");
  video1.mozSrcObject = stream;
--------------------------------------------------------
B2G Desktop ver 2.2 : work well.
B2G Desktop m-c     : not work.

Perhaps this bug block bug 1180589.(see bug 1180589 comment #58)
Blocks: 1180589
Blocks: 1214520
No longer blocks: 1214520
No longer blocks: 1180589
Component: Audio/Video → Audio/Video: Playback
Priority: -- → P2
(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #10)

> No idea why stream capture wouldn't work with MSE. MSE doesn't work at all
> for me on Linux, apparently. Querying jya, who might be able to help.

MSE is now fully supported on Linux (both webm and h264 (if FFmpeg's libavcodec is installed).

Not sure how I could help further here.

Sorry for the delay in answering (if 6 months can be considered a "delay")
Flags: needinfo?(jyavenard)
Mass change P2 -> P3
Priority: P2 → P3
I believe this is the bug report (at least nr 2 from https://bugzilla.mozilla.org/show_bug.cgi?id=1177793#c5) why I'm getting a black screen. I basically want:
==============
var video1 = document.getElementById("video1");
video1.autoplay = true;
video1.src = "https://webrtc.github.io/samples/src/video/chrome.mp4";

... after it started playing ...
  
var stream = video1.mozCaptureStream();
var video2 = document.getElementById("video2");
video2.autoplay = true;
video2.srcObject = stream;
==============

I believe the issue here is that IsSameOrigin is false right? I cannot find this back in https://hg.mozilla.org/mozilla-central/file/tip/dom/media/MediaDecoderStateMachine.cpp but I can still confirm I get a black screen for mozilla 55.

Is there an issue why this cannot be fixed, or is there really a security issue here?

Regards,

Maarten
jib, thoughts?  (baku is on vacation; he'd also be good to look)
Flags: needinfo?(jib)
(In reply to Maarten Breddels from comment #24)
> is there really a security issue hereCT

Note that comment 2 is about allowing cross-origin streams in privileged (chrome) code AFAICT.
If yours is a regular web page, then you're not allowed the bits of that video [1]. The spec [2] says:

    "Media elements can render media resources from origins that differ from the origin of the media element. In those cases, the contents of the resulting MediaStreamTrack MUST be protected from access by the document origin."

It goes on to explain how protection measures differ. Try https://jsfiddle.net/jib1/1kz9hfaL/ to see a couple.

Technically, since the only thing your brief example does with the stream is to play it, there's no issue, but it's also rather pointless, so I assume it's a stand-in for other stream sinks?

Unlike canvas, we don't support "origin-dirty" streams, we just blank them instead. I suppose we could add support for that, and instead blank at the point of contact with sinks like MediaRecorder, RTCPeerConnection and AudioContext.createMediaStreamSource(), but the use-cases seem limited.

[1] https://en.wikipedia.org/wiki/Same-origin_policy
[2] https://w3c.github.io/mediacapture-fromelement/#security-considerations
Flags: needinfo?(jib)
Dear Jan-Ivar,

thanks for the explanation, I did not consider/understand the full security issue, but I think I do now. Basically, you don't want to expose the video pixels that could come from private sources, but displaying them is ok.
I was developing this with google chrome, which does support this (although captureStream is still behind standard disabled flags, chrome 60).
To give you some background, it's part of a library [1] with a model/view concept where the mediastream is the model (implemented for video with an invisible video tag), and you can display it, which is implemented again using a video tag (this time visible). This could be circumvented drawing it to a canvas, but my guess is that would waste cpu cycles. Although it would be of more limited use, since this video would not be possible to share over webrtc, it could be used to draw in a WebGL scene for instance [2,3].

Regards,

Maarten Breddels

[1] https://ipywebrtc.readthedocs.io/en/latest/
[2] https://ipyvolume.readthedocs.io/en/latest/
[3] https://twitter.com/maartenbreddels/status/894983501996584961
Hiding this bug for now while we decide how to handle this. Need to tell the Chrome security team. After that could perhaps just hide the last few comments and un-hide this bug.
Group: media-core-security
Actually, the video->video is not a security bug (since still you cannot get the data). I did notice however, I can send a cross-origin stream over WebRTC, which must be a bug. Feel free to submit the Google Chrome team, or let me know where to do that.
(In reply to Maarten Breddels from comment #27)

Wow, that looks awesome, thanks for sharing! Also, thanks for the info on Chrome. I've updated the fiddle to work there: https://jsfiddle.net/jib1/1kz9hfaL/ - Unfortunately, that looks like a security bug in Chrome.

> This could be circumvented drawing it to a canvas

Actually no, because as the fiddle shows, even in Chrome you'll get:

  canvas getImageData(): SecurityError: Failed to execute 'getImageData' on 'CanvasRenderingContext2D': The canvas
  has been tainted by cross-origin data.
  canvas captureStream(): SecurityError: Failed to execute 'captureStream' on 'HTMLCanvasElement': Canvas is not origin-clean.

But switch s/video1/video2/ here:

   canvas3.getContext('2d').drawImage(video2, 0, 0, 160, 90);

.. and I see two videos and getImageData() now returns [Object ImageData] in Chrome Canary (62), without any chrome flags AFAICT.

Dan will email the Chrome team.
Flags: needinfo?(dveditz)
Both Firefox and Chrome can do:
  canvas3.getContext('2d').drawImage(video1, 0, 0, 160, 90);
So you can mimic video playback from a tainted source to a canvas (by doing a timer etc), but Chrome can do this directly to a HTMLVideoElement, while Firefox cannot (but Chrome does not allow you can do getImageData, which means it is safe). Meaning a workaround to get similar behaviour on Firefox would be this canvas+timer, which isn't very elegant.
Maarten, yes I see what you mean now about simulating playback.

You're also right about the real Chrome sec bug appearing to be webrtc: https://jsfiddle.net/jib1/1kz9hfaL/9/
In light of the use-case in comment 27 we should perhaps consider supporting tainted MediaStreamTracks instead of blanking them.
(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #30)
> (In reply to Maarten Breddels from comment #27)
> 
> But switch s/video1/video2/ here:
> 
>    canvas3.getContext('2d').drawImage(video2, 0, 0, 160, 90);
> 
> .. and I see two videos and getImageData() now returns [Object ImageData] in
> Chrome Canary (62), without any chrome flags AFAICT.

But you don't get any image on the canvas, so it makes sense because it's not tainted yet.

> Dan will email the Chrome team.

It seems a broken or incomplete that you can drawImage() from a regular video but not from a stream captured from a video, but that seems to be what's happening in chrome:
https://jsfiddle.net/dveditz/2t673qov/

Doesn't look like a security bug to me. Letting the second video play and then blocking export seems more useful than not letting the cross-origin play at all. Even allowing the second drawImage() would be OK as long as we propagate the taint correctly. It's certainly reasonable to draw a line somewhere and decide going further is not worth the work, but it would be nice if we and Chrome drew the line in similar places :-)

I haven't looked at your webrtc case yet.
Yeah, looking at the fiddle, the behaviour of the canvas1 is correct.

The capture of video1 to video2 should work though.  That stream should be cross-origin tainted (as discussed).  We taint MediaStreamTrack coming out of WebRTC (if it has a peerIdentity and isolation is requested), so we know that it works.

What should happen in Dan's fiddle is that you get two playing videos at the top.  A screen grab in the middle (of the same frame ideally), and a video that shows that single frame in the bottom.

The media should send black over WebRTC, but we won't find out until capture taints rather than captures black.
> It seems a broken or incomplete that you can drawImage() from a regular video but not from a stream captured from a video, but > that seems to be what's happening in chrome:
> https://jsfiddle.net/dveditz/2t673qov/

You can only drawImage/captureStream after onplayread, video1 was already playing, so that worked, see this fix:
https://jsfiddle.net/j5yxec3b/

Yeah, I think capture from a tainted source and rendering that on a 'save' element (like a video element marked as tainted) would be behaviour as expected (since it doesn't do any harm).

I was also wondering about WebGL, and there Google Chrome has another security issue, you can render a video to a WebGL and read out that canvas. I couldn't set up a small fiddle, but I can show you here:
https://www.astro.rug.nl/~breddels/ipyvolume/tainted/
Note that this also went the video->captureStream->video path, directly passing the original video element to WebGL behaves save.
As Martin mentions in comment 35 we have support for tainted tracks. The blanking [1] predates much of the logic around tainted tracks though.

Removing the blanking shouldn't be much further away than a mochitest or two.

On the other hand, it doesn't look like principal changes for MediaDecoder propagate properly to DecodedStream, thus to individual media chunks, per [2], [3]. That will have to be resolved first for un-tainting a track (it'll still get tainted on a principal change at [4]).


[1] http://searchfox.org/mozilla-central/rev/b258e6864ee3e809d40982bc5d0d5aff66a20780/dom/media/mediasink/DecodedStream.cpp#637
[2] http://searchfox.org/mozilla-central/rev/b258e6864ee3e809d40982bc5d0d5aff66a20780/dom/media/MediaDecoder.cpp#932
[3] http://searchfox.org/mozilla-central/rev/b258e6864ee3e809d40982bc5d0d5aff66a20780/dom/media/MediaDecoderStateMachine.cpp#2800
[4] http://searchfox.org/mozilla-central/rev/b258e6864ee3e809d40982bc5d0d5aff66a20780/dom/html/HTMLMediaElement.cpp#3234
(In reply to Daniel Veditz [:dveditz] from comment #37)
> The Chrome issue is
> https://bugs.chromium.org/p/chromium/issues/detail?id=756877

They used that bug to fix a null-deref crash I found while trying to create an .html file for the testcase. The actual tainting issue was spun off into https://bugs.chromium.org/p/chromium/issues/detail?id=761622
I cannot see those chromium issues, can I get access to them, and will that issue in this one be public once fixed?
I'll unhide this when Google unhides their bug.

Do you have an account on their bug tracker? If so I can ask that they add you to it.
Keywords: sec-other
Whiteboard: Keep hidden until Chrome bug is fixed
Thanks, that would be maartenbreddels _ gmail.com

The Chrome bugs were fixed in Chrome 62... we can unhide this now. Since we were correctly propagating the cross-origin taint and now Chrome's behavior matches we're not going to "fix" this as requested

Group: media-core-security
Status: NEW → RESOLVED
Closed: 7 months ago
Resolution: --- → INVALID

I think the request here was still valid. I believe we still output black rather than tainted video in the face of tainted input. We should try to match what Chrome does here for local playback, which was being discussed here I think. Reopening for consideration.

Came across this while trying to figure out why https://webrtc.github.io/samples/src/content/capture/video-pc/ emits black in Firefox but not Chrome.

Status: RESOLVED → REOPENED
Resolution: INVALID → ---

I updated your old fiddle for a manual test case.

I have a patch for this but it needs tests.

Depends on D36917

Pushed by pehrsons@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/18a62a05982e
Pass principal instead of forcing black when capturing cross-origin media resource. r=jya,jib
https://hg.mozilla.org/integration/autoland/rev/46ad935ecd17
Modernize test_streams_capture_origin.html. r=jib
https://hg.mozilla.org/integration/autoland/rev/f775de3b67f6
Update mochitest. r=jib
Status: REOPENED → RESOLVED
Closed: 7 months ago2 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70
Assignee: nobody → apehrson
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.