mozCaptureStream only returns MediaStreamTracks the first time called before src is changed on <video> element then returns empty array with no MediaStreamTrack elements

RESOLVED FIXED in Firefox 68

Status

()

defect
P2
normal
RESOLVED FIXED
2 months ago
2 months ago

People

(Reporter: guest271314, Assigned: pehrsons)

Tracking

(Regression)

68 Branch
mozilla68
Points:
---

Firefox Tracking Flags

(geckoview66 wontfix, firefox-esr60 unaffected, firefox66 wontfix, firefox67 wontfix, firefox68 fixed)

Details

Attachments

(3 attachments)

Reporter

Description

2 months ago

User Agent: Mozilla/5.0 (X11; Linux i686; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

  1. Set src of HTML <video> element to a media resource
  2. Execute mozCaptureStream on <video> element
  3. At pause event of <video> element repeat 1.

Actual results:

  1. MediaStream instance returned the first occasion mozCaptureStream is executed is populated by an audio and video MediaStreamTrack
  2. MediaStream instance returned for each subsequent execution of mozCaptureStream after setting src of <video> element to a different media resource is not populated with any MediaStreamTrack, and empty array is returned for MediaStream.getTracks()

Expected results:

  1. MediaStream returned by mozCaptureStream should be populated by an audio and video MediaStreamTrack each occasion mozCaptureStream is executed on the same HTML <video> after setting the src to a different media resource
Assignee

Comment 1

2 months ago

Here's an STR that works with your testcase:
1 Click the text "Click"
2 See that (several) videos visibly play, monitor console output

Expected: No errors or asserts in console

Actual: TypeError: playlistVideoTrack is undefined (and more)

I debugged a bit under rr. This seems like a regression from https://hg.mozilla.org/mozilla-central/rev/243a33803d16 (bug 1423241). When a media element already has output streams attached, they're added in FinishDecoderSetup() [1], before metadata is available. Later, tracks are only added prior to playing as a stream is attached to a decoder where no others are attached, in EnsureOutputStreamManager [2], [3].

We could either delay the attaching of existing output streams from [1] to when metadata is available in HTMLMediaElement; or add output tracks to OutputStreamManager automatically when metadata is available in MediaDecoder, so that the order of the calls from HTMLMediaElement doesn't matter.

The former is simpler, the latter is better. I'll have to check that we're ok wrt TrackID collisions if we do the latter though.

Also, this needs a testcase.

[1] https://searchfox.org/mozilla-central/rev/0376cbf447efa16922c550da3bfd783b916e35d3/dom/html/HTMLMediaElement.cpp#4573
[2] https://searchfox.org/mozilla-central/rev/1b2636e8517aa48422ed516affe4d28cb7fa220a/dom/media/MediaDecoder.cpp#250
[3] https://searchfox.org/mozilla-central/rev/1b2636e8517aa48422ed516affe4d28cb7fa220a/dom/media/MediaDecoderStateMachine.cpp#3805-3815

Assignee: nobody → apehrson
Status: UNCONFIRMED → NEW
Component: Untriaged → Audio/Video: Playback
Ever confirmed: true
Priority: -- → P2
Product: Firefox → Core
Regressed by: 1423241
Assignee

Comment 3

2 months ago

This moves the responsibility for creating MediaStreamTracks from
DecodedStream::Start to MediaDecoder. This let's MediaDecoder create them as
soon as metadata is known. This gives the application guarantees on when tracks
can be expected to exist, and aligns with the spec that says they should be
created when metadata is known.

Comment 4

2 months ago
Pushed by pehrsons@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/284e4c610279
Add mochitest. r=padenot
https://hg.mozilla.org/integration/autoland/rev/eab0e21e5347
Always pre-create MediaStreamTracks for DecodedStream in MediaDecoder. r=padenot

Comment 5

2 months ago
bugherder
Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.