Offer downscaled resolutions and decimated framerates in getUserMedia.

NEW
Assigned to

Status

()

Core
WebRTC: Audio/Video
P3
normal
Rank:
25
a year ago
a month ago

People

(Reporter: jib, Assigned: mchiang)

Tracking

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox50 affected)

Details

Attachments

(2 obsolete attachments)

After making constraints respect concurrent gUM access (bug 1213397), and seeing the issues with averaging and other attempts at resolving conflicts (e.g. bug 1213517 comment 152), plus the fact that we seem to mix FPS with maxFPS in our implementations (bug 1273734), it's becoming clear that we need to consider offering up downscaled resolutions and decimated framerates.

This would be similar to what Chrome does, though we need to answer whether we wish to crop or always preserve aspect (Chrome crops).
Depends on: 1213517

Updated

a year ago
Rank: 25
Priority: -- → P2
See Also: → bug 1305018
(Reporter)

Updated

10 months ago
See Also: → bug 1340826
(Reporter)

Updated

10 months ago
Duplicate of this bug: 1340826

Comment 2

10 months ago
My preference would be to work the same as chrome, i.e. cropping, which I think is preferable to getting black bars.

The typical use case is that the site requests a 4:3 aspect ratio resolution, and the webcam is widescreen. In this case it is preferable to get the cropped 4:3 image rather than one with black bars at the top and bottom.
(Reporter)

Updated

7 months ago
See Also: → bug 1359406
E.g. Jitsi could ask for { video: { width: 1280, height: 720 }, audio: true }

and it should work well for their needs, defaulting to 1280x720 or the closest to it.

It would even prompt Firefox users once for cam+mic instead of twice for each like they do now.

[1] https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia#Parameters
(Reporter)

Updated

4 months ago
Duplicate of this bug: 1385578
(Assignee)

Updated

4 months ago
Assignee: nobody → mchiang
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
(Assignee)

Comment 7

4 months ago
These patches are only WIP.
I would need more time to verify these patches on different platform and scenarios.
I am also working on decimated framerate part.
(Assignee)

Comment 8

4 months ago
mozreview-review
Comment on attachment 8892951 [details]
Bug 1286945 - support getusermedia continuous constraint by down scaling.

https://reviewboard.mozilla.org/r/163962/#review169276

::: dom/media/systemservices/CamerasChild.h:187
(Diff revision 1)
>                     const int capture_id, webrtc::VideoCaptureCapability& capability,
> +                   webrtc::VideoCaptureCapability& constraint,

We want to separate the two different concepts:
1) the configuration we set to the hardware (capability)
2) the constraint user sets (constraint)

For example, user may request a resolution 1000x700, which is not supported by the camera hardware.

We then config camera with the capability 1280x720.

Then downscale the output frame to the constraint 1000x700.
(Assignee)

Updated

4 months ago
Attachment #8892950 - Attachment is obsolete: true
Attachment #8892950 - Flags: review?(jib)
(Assignee)

Updated

4 months ago
Attachment #8892951 - Attachment is obsolete: true
Attachment #8892951 - Flags: review?(jib)
(Assignee)

Updated

4 months ago
Depends on: 1388219
Mass change P2->P3 to align with new Mozilla triage process.
Priority: P2 → P3
Current front-runner on behavior is to only downscale when we'd otherwise fail with OverconstrainedError, as described in bug 1388667 comment 10.
You need to log in before you can comment on or make changes to this bug.