1340826 - getUserMedia constraints don't work with newer FaceTime HD cameras

Status: RESOLVED DUPLICATE of bug 1286945

(Core :: WebRTC: Audio/Video, defect, P1)

Description

[David Jameson]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36

Steps to reproduce:

Using a 2016 Macbook Pro 13in:

1. Go to https://jsfiddle.net/z2cj12k4/33/ in firefox
2. Click start.


Actual results:

Gives error "constraints could not be satisfied".


Expected results:

Should be successful. On an older (early 2011) Macbook pro 13in with a FaceTime HD camera, it works perfectly, and you can specify constraints such as:

{ video: {
  width: {  max: 160 },
  height: { max: 120 },
} };

and it will work perfectly. For some reason Firefox only permits 1280x720 resolution with the newer FaceTime HD cameras. With the older Facetime HD camera, it seems to default to 640x480 and allows other, lower resolutions.

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

I suspect this is Yet Another Case of misunderstanding the spec, which does not guarantee that the browser will scale and hide the real camera capabilities (as Chrome does).  The spec invites browsers to expose the capabilities such that the entire complex Constraints language has a purpose and gives browsers the ability to interact with the real resolutions the camera supports.

Comment 2

[David Jameson]

The problem is that doing the minimum (only returning 1280x720) isn't terribly useful. On the same machine, chrome returns any resolution requested, and an older version of the macbook pro (with facetime HD camera) also returns any resolution requested.

It's an issue when transmitting video data, as 1280x720 will use a lot more bandwidth than lower resolutions. (We'd prefer not to specify a hard bitrate limit). Firefox/webrtc already has to re-code the video using the vp8 codec, so changing the resolution probably doesn't add any significant extra processing.

Firefox does already seem to hide and scale the real camera capabilities. If you try a 2011 macbook pro you'll see firefox delivers any resolution requested.


Is there any easy way to check (outside firefox) what native resolutions are supported by the facetime hd camera?

Comment 3

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

I have a 2016 MBP 15" and I'm not seeing this problem. Do we know if the camera on the 13" is different?

FWIW my "About This Mac"/"System Report..." on my working MBP 15" says:

  FaceTime HD Camera:
    Model ID:    UVC Camera VendorID_1452 ProductID_34304

Firefox does not rescale cameras, rather we get capabilities from AVFoundation (since Firefox 49 - bug 1180725. Prior to 49 we used the now-discontinued QTKit API which rescaled behind our backs).

To dump camera capabilities we get from AVFoundation in Firefox, follow these steps in Terminal:

 1. Type: export NSPR_LOG_MODULES=MediaManager:4
 2. Type: export NSPR_LOG_FILE=/tmp/nspr.log
 3. Run: /Applications/Firefox.app/Contents/MacOS/firefox

and examine or upload the produced log file named something like /tmp/nspr.log.child-1

This information would be helpful, as I don't know of anyone on the team who has a 13" MBP.

If 1280x720 is the only mode your camera supports, then jesup is correct in comment 1.

Note that you can scale down video on send in Firefox using

     sender.setParameters({encodings: [{scaleResolutionDownBy: fromHeight / toHeight}]})

See https://jsfiddle.net/jib1/kv14z7r6/

Comment 4

[David Jameson]

My camera seems to be different:

FaceTime HD Camera:

  Model ID:	Apple Camera VendorID_0x106B ProductID_0x1570
  Unique ID:	CC27034Q5HJGJJM5N

I had to use NSPR_LOG_MODULES=MediaManager:5 to get the info. It looks like it is just reported 1280x720. See attachment.

Comment 5

[David Jameson]

Attachment: nspr.log.child-1

Comment 6

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

> D/MediaManager ChooseCapability: prefs: 640x480 @30-10fps
> D/MediaManager Constraints: width: { min: -2147483647, max: 2147483647 }
> D/MediaManager              height: { min: -2147483647, max: 2147483647 }
> D/MediaManager              frameRate: { min: -inf, max: inf, ideal: 30.000000 }
> D/MediaManager Capability: 1280 x  720 x 30 maxFps, YUY2, Unknown codec. Distance = 0
> D/MediaManager Capability: 1280 x  720 x 30 maxFps, UYVY, Unknown codec. Distance = 0
> D/MediaManager Capability: 1280 x  720 x 30 maxFps, NV12, Unknown codec. Distance = 0
> D/MediaManager Chosen capability: 1280 x  720 x 30 maxFps, YUY2, Unknown codec. Distance = 0

Wow, doesn't even list 640x480. Munro, do you think this could be an AVFoundation bug, maybe other modes we're not seeing/supporting, or is this all this new camera will produce we think?

David, you could perhaps try playing something back in iMovie and see what resolution choices it gives you?

Comment 7

[David Jameson]

In iMovie when I create a movie and choose share->file there are two options for resolution: 720p and 540p. If there is something else you meant, let me know. I don't see any way of playing back an existing movie in iMovie (although this is the first time I've used it, so I'm not very familiar with it).

Comment 8

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Thanks David, yeah that may be playback and unrelated. Just an idea. Still, no 640x480 though. Unless there's a bug covering up other modes, this may just be what your camera offers.

Comment 9

[Andreas Pehrson [:pehrsons]]

Making this a low P1 until we figure out where the issue is. Please adjust accordingly.

Comment 10

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

I just noticed my new 2016 MBP camera has dropped all modes below 640x480. While not as aggressive as David's case, it's enough to suggest a hardware trend to me, and that the hardware information is probably correct. Assuming this is true, our software works as expected in response.

I'm closing this as a dup of the request to downscale resolutions. I'll leave the ni? to Munro open in case he knows otherwise.

See my blog post [1] for a solution to comment 2.

[1] https://blog.mozilla.org/webrtc/fiddle-week-downscale-video-peerconnection/

Comment 11

[Munro Mengjue Chiang [:mchiang] Be Mozillian since 1/19/2018]

(In reply to Jan-Ivar Bruaroey [:jib] from comment #10)
> I just noticed my new 2016 MBP camera has dropped all modes below 640x480.
> While not as aggressive as David's case, it's enough to suggest a hardware
> trend to me, and that the hardware information is probably correct. Assuming
> this is true, our software works as expected in response.
> 
> I'm closing this as a dup of the request to downscale resolutions. I'll
> leave the ni? to Munro open in case he knows otherwise.
> 
> See my blog post [1] for a solution to comment 2.
My new 2016 MBP 15" also offered 720p and VGA only.
But when I plug in my external Logitech c920r, it still offers resolutions between 160x90 ~ 1080p.
It seems there is a trend that the newer MBP built-in camera module will offer fewer and fewer native resolutions, which makes downscale & drop frame mechanism more important now.