Video MediaStreamTrack.getSettings returns empty object with media.navigator.streams.fake = true
Categories
(Core :: WebRTC, defect, P3)
Tracking
()
Tracking | Status | |
---|---|---|
firefox69 | --- | fixed |
People
(Reporter: devin.wilson, Assigned: pehrsons)
References
Details
Attachments
(13 files)
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review | |
47 bytes,
text/x-phabricator-request
|
Details | Review |
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36
Steps to reproduce:
- Open Firefox 66
- Navigate to
about:config
and set the "media.navigator.streams.fake" option to true - Get a video track using getUserMedia
const stream = await navigator.mediaDevices.getUserMedia({ video: true });
- Get the track settings
`const settings = stream.getTracks()[0].getSettings();
Actual results:
settings is an empty object:
{}
This makes it harder to have automated tests validate the resolution of a stream without a video element.
Expected results:
settings should be an object with resolution and frame rate:
{
frameRate: 30,
height: 480,
width: 640
}
Updated•6 years ago
|
Comment 2•6 years ago
•
|
||
Unfortunately, we don't support constraints in our fake devices. This is a long standing issue. See bug 1088621.
Updated•6 years ago
|
Reporter | ||
Comment 5•6 years ago
|
||
Would it be possible to at least expose the actual dimensions of the stream even if you can't use constraints to request a specific resolution? Is it difficult to add constraints and/or settings support to the fake devices or is it more a matter of priority?
Assignee | ||
Updated•6 years ago
|
Assignee | ||
Comment 6•6 years ago
|
||
Assignee | ||
Comment 7•6 years ago
|
||
This will help our WPT stats.
Depends on D32960
Assignee | ||
Comment 8•6 years ago
|
||
Depends on D32961
Assignee | ||
Comment 9•6 years ago
|
||
Depends on D32962
Assignee | ||
Comment 10•6 years ago
|
||
Depends on D32963
Assignee | ||
Comment 11•6 years ago
|
||
Depends on D32964
Assignee | ||
Comment 12•6 years ago
|
||
Depends on D32965
Assignee | ||
Comment 13•6 years ago
|
||
This is not mandated behavior by the spec.
Depends on D32966
Updated•6 years ago
|
Assignee | ||
Comment 14•6 years ago
|
||
Depends on D32965
Updated•6 years ago
|
Assignee | ||
Comment 15•6 years ago
|
||
Depends on D32967
Assignee | ||
Comment 16•6 years ago
|
||
Assignee | ||
Comment 17•6 years ago
|
||
Depends on D33470
Assignee | ||
Comment 18•6 years ago
|
||
Depends on D33472
Updated•6 years ago
|
Comment 19•6 years ago
|
||
Comment 20•6 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/f1b4c01cf3b9
https://hg.mozilla.org/mozilla-central/rev/ec41532452ed
https://hg.mozilla.org/mozilla-central/rev/257a8501db35
https://hg.mozilla.org/mozilla-central/rev/4f3d4d29aaa3
https://hg.mozilla.org/mozilla-central/rev/e046f15e4dbd
https://hg.mozilla.org/mozilla-central/rev/1e0ef2e38659
https://hg.mozilla.org/mozilla-central/rev/3a46c2b53953
https://hg.mozilla.org/mozilla-central/rev/c0f5518234b9
https://hg.mozilla.org/mozilla-central/rev/0620e8b80b92
https://hg.mozilla.org/mozilla-central/rev/04b0f6e76803
https://hg.mozilla.org/mozilla-central/rev/4be44814ed94
https://hg.mozilla.org/mozilla-central/rev/a1d860f55aab
https://hg.mozilla.org/mozilla-central/rev/fd8eaee557a5
Comment 21•6 years ago
|
||
Firefox 69 getSettings() called on remote WebRTC MediaStreamTrack and MediaStreamTrack from HTMLMediaElement.captureStream() return an empty object. getConstraints() returns Object { mediaSource: "camera" }
. Is that the expected behaviour?
Assignee | ||
Comment 22•6 years ago
|
||
(In reply to guest271314 from comment #21)
Firefox 69 getSettings() called on remote WebRTC MediaStreamTrack and MediaStreamTrack from HTMLMediaElement.captureStream() return an empty object.
Expected.
getConstraints() returns
Object { mediaSource: "camera" }
. Is that the expected behaviour?
Not expected. I see this in release too. Filed bug 1560907.
Comment 23•6 years ago
|
||
@(In reply to Andreas Pehrson [:pehrsons] from comment #22)
(In reply to guest271314 from comment #21)
Firefox 69 getSettings() called on remote WebRTC MediaStreamTrack and MediaStreamTrack from HTMLMediaElement.captureStream() return an empty object.
Expected.
getConstraints() returns
Object { mediaSource: "camera" }
. Is that the expected behaviour?Not expected. I see this in release too. Filed bug 1560907.
Why does getSettings()
for a MediaStreamTrack
from <video>.captureStream()
not provide at least the width
and height
of the underlying image (frame)? Why is an empty object expected?
Comment 24•6 years ago
|
||
Chromium 75-77 does provide details of the MediaStreamTrack
when getSettings()
is executed.
Assignee | ||
Comment 25•6 years ago
|
||
We don't implement getSettings() for other tracks than gUM tracks. The method is exposed but internally it's empty. If you want a workaround you can look at canvas.width
and canvas.height
instead.
Comment 27•6 years ago
|
||
(In reply to Andreas Pehrson [:pehrsons] from comment #25)
We don't implement getSettings() for other tracks than gUM tracks.
Why? Is Mozilla in accord with the specifications relevant to not implementing getSettings()
for other than gUM tracks?
Assignee | ||
Comment 28•6 years ago
|
||
We don't implement it right now. Patches welcome!
Comment 29•6 years ago
|
||
have not yet learned C++ or Rust, else would have already gotten rid of mozCaptureStream()
, removed restriction on MediaRecorder
stopping when <video>
src
changed, and ported mkvmerge
to JavaScript, just for the utility. Perhaps it is now time to finally dive in to a language other than JavaScript.
Comment 30•6 years ago
|
||
(In reply to Andreas Pehrson [:pehrsons] from comment #28)
We don't implement it right now. Patches welcome!
Recommended reading for contributing to Mozilla source code (to write patches)?
Assignee | ||
Comment 31•6 years ago
|
||
Comment 35•6 years ago
|
||
(In reply to guest271314 from comment #27)
(In reply to Andreas Pehrson [:pehrsons] from comment #25)
We don't implement getSettings() for other tracks than gUM tracks.
Why? Is Mozilla in accord with the specifications relevant to not implementing
getSettings()
for other than gUM tracks?
Note the mediacapture-main spec says: "The constraints specified in this specification apply only to instances of MediaStreamTrack generated by getUserMedia(), unless stated otherwise in other specifications."
Comment 36•6 years ago
|
||
(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #35)
(In reply to guest271314 from comment #27)
(In reply to Andreas Pehrson [:pehrsons] from comment #25)
We don't implement getSettings() for other tracks than gUM tracks.
Why? Is Mozilla in accord with the specifications relevant to not implementing
getSettings()
for other than gUM tracks?Note the mediacapture-main spec says: "The constraints specified in this specification apply only to instances of MediaStreamTrack generated by getUserMedia(), unless stated otherwise in other specifications."
Note that MediaStreamTrack
at Media Stream Recording specification https://www.w3.org/TR/mediastream-recording/#overview
- Overview
This API attempts to make basic recording very simple, while still allowing for more complex use cases. In the simplest case, the application instantiates a MediaRecorder object, calls start() and then calls stop() or waits for the MediaStreamTrack(s) to be ended.
and Media Capture from DOM Elements https://www.w3.org/TR/mediacapture-fromelement/#dfn-mediastreamtrack
- HTML Media Element Media Capture Extensions
Method captureStream()is defined on HTML [HTML5] media elements. Methods for capture are added to both HTMLMediaElement and HTMLCanvasElement.
Both MediaStream and HTMLMediaElement expose the concept of a track. Since there is no common type used for HTMLMediaElement , this document uses the term track to refer to either VideoTrack or AudioTrack. MediaStreamTrack is used to identify the media in a MediaStream.
both links to Media Capture and Stream specification https://www.w3.org/TR/mediacapture-streams/#mediastreamtrack which means that "unless otherwise stated" at Media Stream Recording directly incorporates by reference MediaStreamTrack
at Media Capture and Streams. thus getSettings()
should return the relevant constraints of a MediaStreamTrack
when captureStream()
is executed to get the MediaStreamTrack
.
Comment 37•6 years ago
•
|
||
(In reply to guest271314 from comment #36)
both links to Media Capture and Stream specification https://www.w3.org/TR/mediacapture-streams/#mediastreamtrack which means that "unless otherwise stated" at Media Stream Recording directly incorporates by reference
MediaStreamTrack
at Media Capture and Streams. thusgetSettings()
should return the relevant constraints of aMediaStreamTrack
whencaptureStream()
is executed to get theMediaStreamTrack
.
"unless otherwise stated" is not a quote from anywhere I can find. Yes, those specifications normatively say the getSettings()
method must be implemented for tracks of those sources, but do not say to return anything, or to implement any constrainable properties for those sources.
This used to be under-specified, but got the stronger mediacapture-main language recently, because implementers need specific instructions on how to implement constraints for a specific source, since it's not always obvious how to do so. That's why mediacapture-main now admonishes each spec to spell out not only the constraints an implementer is required to support, but also how they must work.
There are many examples of constraints unique to a source (e.g. facingMode
), and even where there's overlap (e.g. width
) there can be subtle differences (screen capture tracks, for instance, must not crop because that might lose important info and/or window borders and look bad, whereas camera tracks were (and still are) allowed to crop to create mock 16:9 out of 4:3 etc. Also resizeMode
isn't "none"
on retina displays.)
Another example is remote peer connection tracks, where there's questions around whether lowering width
should impact the network congestion algorithm, or whether resizeMode = "none"
would guarantee the dimensions sent by the other peer (probably not, because that's quite literally action at a distance). So right now constraints on remote tracks might be left to an extension spec, even though Chrome has some support.
Comment 38•6 years ago
|
||
(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #37)
(In reply to guest271314 from comment #36)
both links to Media Capture and Stream specification https://www.w3.org/TR/mediacapture-streams/#mediastreamtrack which means that "unless otherwise stated" at Media Stream Recording directly incorporates by reference
MediaStreamTrack
at Media Capture and Streams. thusgetSettings()
should return the relevant constraints of aMediaStreamTrack
whencaptureStream()
is executed to get theMediaStreamTrack
."unless otherwise stated" is not a quote from anywhere I can find. Yes, those specifications normatively say the
getSettings()
method must be implemented for tracks of those sources, but do not say to return anything, or to implement any constrainable properties for those sources.This used to be under-specified, but got the stronger mediacapture-main language recently, because implementers need specific instructions on how to implement constraints for a specific source, since it's not always obvious how to do so. That's why mediacapture-main now admonishes each spec to spell out not only the constraints an implementer is required to support, but also how they must work.
There are many examples of constraints unique to a source (e.g.
facingMode
), and even where there's overlap (e.g.width
) there can be subtle differences (screen capture tracks, for instance, must not crop because that might lose important info and/or window borders and look bad, whereas camera tracks were (and still are) allowed to crop to create mock 16:9 out of 4:3 etc. AlsoresizeMode
isn't"none"
on retina displays.)Another example is remote peer connection tracks, where there's questions around whether lowering
width
should impact the network congestion algorithm, or whetherresizeMode = "none"
would guarantee the dimensions sent by the other peer (probably not, because that's quite literally action at a distance). So right now constraints on remote tracks might be left to an extension spec, even though Chrome has some support.
Informative reply. Am particularly interested in width
and height
constraints relevant to all variations of a MediaStreamTrack
of kind
"video"
. While Chromium does output width
and height
for getSettings()
after captureStream()
from a DOM
element, Chromium still has issues both outputting the width
and height
at <video>
element and recording the width
and height
at MediaRecorder
implementation https://bugs.chromium.org/p/chromium/issues/detail?id=972470, https://bugs.chromium.org/p/chromium/issues/detail?id=983777. Additionally, when using WebRTC
and replaceTrack()
the first video track is set to the default value of the <video>
element at https://github.com/guest271314/MediaFragmentRecorder/tree/webrtc-replacetrack and the branched variations. For the MediaRecorder
specification re width
and height
filed this PR https://github.com/w3c/mediacapture-record/pull/172. Your comment appear to indicate accompanying PR's should be filed to specify how width
and height
are determined relevant to the output WebM file. That is, the output width
and height
should match the values of getSettings()
.
Description
•