1305018 - getUserMedia returns stream of non-ideal resolution since Firefox 49

Status: RESOLVED INCOMPLETE

(Core :: WebRTC: Audio/Video, defect)

Description

[Wang Guan]

Attachment: gum-example.html

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:49.0) Gecko/20100101 Firefox/49.0
Build ID: 20160916101415

Steps to reproduce:

Open attached gum-example.html (a minimal example) with Firefox 49.

My only camera was the `FaceTime HD camera` one in MacBook Pro (Retina, 15-inch, Mid 2015).


Actual results:

navigator.mediaDevices.getUserMedia returns video stream of 640x480, not obeying ideal width or height or aspect ratio.


Expected results:

With identical constraints, getUserMedia have been returning stream of 1024x576 until Firefox 48.

Comment 1

[Wang Guan]

Another problem: when I specify height constraint `{min:540}` instead of `{ideal:540}`, getUserMedia returns stream of 1280x720, which is much higher than I wanted.

Comment 2

[Byron Campen [:bwc]]

Current nightly on my macbook pro gives a 960x540 capture, which is exactly what the test script asks for. Is nightly behaving better for you?

Comment 3

[Wang Guan]

Hello Byron,

Nightly v52.0a1 is behaving the same.

Is there some way I can provide more detailed log for this problem?

Comment 4

[Byron Campen [:bwc]]

So the test case attached to this page gets you a 640x480 capture on nightly OS X? Or is some detail different?

Comment 5

[Wang Guan]

Hi Byron,

Yes, the attached page returns 640x480 stream in both Firefox 49.0 and nightly v51.0a2.

The same page was returning 1024x576 in Firefox 48 (not exactly matching constraints, but in my case it's good enough).

(All this are tested in OSX 10.11).

Comment 6

[Wang Guan]

Sorry Byron, I was listing an incorrect version.

The 640x480 stream happens in release (49.0), Developer Edition (51.0a2) and nightly (52.0a1 (2016-09-26)) .

Comment 7

[Wang Guan]

We had a few more tests.


Platforms where 640x480 was returned:

- Win10 VirtualMachine / Firefox 49 / FaceTime HD camera
- OSX 10.11 / Firefox 49 / FaceTime HD camera


Platforms where stream fit constraint better:

- OSX 10.11 / Firefox 49 / Logi C920  (1024x576)
- Win 10 / Firefox 49 / Logi C270     (960x544)

Can this issue be related to the FaceTime camera?

Comment 8

[Byron Campen [:bwc]]

I have a 2013 MacBook Pro, and my built-in camera is working fine on OS 10.11.6. It is possible that your camera is behaving differently somehow.

jib, do you have any ideas for how to debug this?

Comment 9

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Yes, add the following environment variables:

    NSPR_LOG_MODULES=MediaManager:5
    NSPR_LOG_FILE=/tmp/gum.log

and do a run. You should get a list with each of the available modes of your camera, and their fitness distance from your ideal constraints.

On OSX, as of 49 we use AVFoundation (bug 1180725) which exposes actual capabilities of the camera, whereas before (with QTkit) we were cheating with a fixed set of hardcoded values regardless of camera, so the new behavior is likely more correct.

Note that getUserMedia in Firefox is a camera discovery API, like in the spec. This means the results you get depend on the user's camera.

Comment 10

[Wang Guan]

Attachment: gum log: OSX 10.11 / Firefox 48

Comment 11

[Wang Guan]

Attachment: gum log: OSX 10.11 / Firefox 49

Comment 12

[Wang Guan]

Thanks jib and Byron, I'm attaching 2 logs from fx48 and fx49.

The 2 log files contains same sequence: start firefox - open gum-example page - approve use of camera - close tab and firefox.

Comment 13

[Randell Jesup [:jesup] (needinfo me)]

It looks as though the camera really only (natively) supports 3 resolutions: 320x240, 640x480, and 1280x720.  This isn't too surprising.

The old way we accessed the camera gave us no knowledge of the actual camera resolution - the MacOS scaled everything to match the request - so we made up a wide range of "standard" resolutions and let the OS scale if they weren't natively supported.  The spec always wanted us to provide the real resolutions; only MacOS scales in the OS in this way.

Because the requested ideal rate falls "between" two resolutions in the camera, 640x480 ends up being a little closer due to the spec-mandated algorithm.  960x540 is a non-standard-ish resolution.  You could bump the ideal values up a little so it's closer to 1280x720 than 640x480.  You would have had the same problem on Windows and Linux with a camera with limited modes, so this would be good to do in any case.  You can use the fitness distance numbers in that log to see which is closer when you try other values.  They're close already (444 vs 500)

Comment 14

[Nils Ohlmeier [:drno]]

Nothing actionable here.