Closed Bug 1150539 Opened 5 years ago Closed 5 years ago

getUserMedia activating camera light, but not receiving video -- audio working OK.

Categories

(Core :: WebRTC: Audio/Video, defect)

x86_64
Windows 7
defect
Not set

Tracking

()

VERIFIED FIXED
mozilla40
Tracking Status
firefox37 --- unaffected
firefox38 --- wontfix
firefox38.0.5 + fixed
firefox39 + verified
firefox40 --- fixed
firefox-esr38 39+ verified
relnote-firefox --- 39+

People

(Reporter: bobowen, Assigned: jib)

References

Details

Attachments

(12 files, 2 obsolete files)

9.32 KB, text/plain
Details
95.66 KB, text/plain
Details
41.73 KB, text/plain
Details
42.20 KB, text/plain
Details
3.06 KB, text/plain
Details
2.58 KB, text/plain
Details
1.18 KB, text/plain
Details
1.22 KB, text/plain
Details
1.06 KB, text/plain
Details
6.06 MB, text/plain
Details
3.71 KB, patch
jib
: review+
Details | Diff | Splinter Review
1.27 KB, patch
jib
: review+
Details | Diff | Splinter Review
Using my Hello conversation, other caller connected OK and could hear me, but not see me.
The indicator light on my camera was on, I had no self view.
Incoming video and audio was fine.

Latest Windows x86 Nightly built from:
https://hg.mozilla.org/mozilla-central/rev/37ddc5e2eb72

Browser console log attached.
Two quick questions:
1) Does this happen every time, or intermittently?
2) To help isolate whether this is a Hello issue or a platform issue, can you see whether you can reproduce it on <http://mozilla.github.io/webrtc-landing/gum_test.html> ?
Flags: needinfo?(bobowen.code)
(In reply to Adam Roach [:abr] from comment #1)
> Two quick questions:
> 1) Does this happen every time, or intermittently?

Every time.

> 2) To help isolate whether this is a Hello issue or a platform issue, can
> you see whether you can reproduce it on
> <http://mozilla.github.io/webrtc-landing/gum_test.html> ?

When I click the "Video" button, I get asked to share the camera and when allowed it says "Success!", but I see no video.
The camera indicator is on.

Camera is working with Vidyo.
Flags: needinfo?(bobowen.code)
Component: Client → WebRTC: Audio/Video
Product: Loop → Core
Bob -- can you provide as much information as possible about your camera? Manufacturer, model, driver name, USB ID (cf. https://msdn.microsoft.com/en-us/library/windows/hardware/ff560019%28v=vs.85%29.aspx and http://digital.ni.com/public.nsf/allkb/335A90747734097886257070006415B9) -- actually, including all of the information from USBView would be helpful.
Summary: Hello - no outgoing video or asking for permission to share, audio working OK. → getUserMedia activating camera light, but not receiving video -- audio working OK.
(In reply to Adam Roach [:abr] from comment #3)
> Bob -- can you provide as much information as possible about your camera?
> Manufacturer, model, driver name, USB ID (cf.
> https://msdn.microsoft.com/en-us/library/windows/hardware/ff560019%28v=vs.
> 85%29.aspx and
> http://digital.ni.com/public.nsf/allkb/335A90747734097886257070006415B9) --
> actually, including all of the information from USBView would be helpful.

Yeah, no problem, it might have to wait until Tuesday now, but I'll see if I can get it done before then.
Attached file USBViewAll.txt
Had more time than I thought.

This is the integrated camera on my Lenovo W540 laptop.

Manufacturer: SunplusIT
model: can't find this anywhere
Driver Provider: SunplusIT
Driver Date: 18/03/2013
Driver Version: 3.4.7.32
Driver Signer: Microsoft Windows Hardware Compatibility Publisher
Just realised that probably included more data than was needed.
Also thought that the output might be different when Hello was running, which it seems to be.
Awesome, thanks. I'm pinging Jesup to see if he's familiar with any other problems like this one.
Flags: needinfo?(rjesup)
We need details about the formats supported and selected for the camera:

run with NSPR_LOG_MODULES=getusermedia:5,mediamanager:5,webrtc_trace:65535 WEBRTC_TRACE_FILE=nspr NSPR_LOG_FILE=whatever

The try gum_test.html for Video and NI me back
Flags: needinfo?(rjesup) → needinfo?(bobowen.code)
Attached file bug1150539nsprlog.txt
*.child-1 log file was empty
Flags: needinfo?(bobowen.code) → needinfo?(rjesup)
This time using the actual test you asked for. :-)

Only stuff in the child file this time.
I don't see any of the webrtc_trace logs there - did you set WEBRTC_TRACE_FILE=nspr ?  If not, it would have saved the log file in WebRTC.log somewhere (perhaps the current directory...)

It's selecting 848x480@15fps (sigh) with type 2, which should work, so we need the trace logs.
Flags: needinfo?(rjesup) → needinfo?(bobowen.code)
(In reply to Randell Jesup|PTO until Apr 6 [:jesup] from comment #12)
> It's selecting 848x480@15fps (sigh) with type 2, which should work, so we
> need the trace logs.

What an odd mode to select when the default 640x480 seems available. Hoping I don't have an algorithm bug, and makes me realize we're not logging constraints used.
Win-build w/constraints-logging available here soon: https://treeherder.mozilla.org/#/jobs?repo=try&revision=154962707b57
(In reply to Jan-Ivar Bruaroey [:jib] from comment #13)
> Created attachment 8588121 [details] [diff] [review]
> log constraints used in MediaManager:5
> 
> (In reply to Randell Jesup|PTO until Apr 6 [:jesup] from comment #12)
> > It's selecting 848x480@15fps (sigh) with type 2, which should work, so we
> > need the trace logs.
> 
> What an odd mode to select when the default 640x480 seems available. Hoping
> I don't have an algorithm bug, and makes me realize we're not logging
> constraints used.

Smells like an algorithm bug perhaps predicated on one of the values being exactly the default (prefs) size
It's not unusual to support 640x480. Does Hello perhaps constraints? Smells like 16:9 math.
Sorry, been away helping my step-daughter move house and she has no broadband yet.

(In reply to Randell Jesup|PTO until Apr 6 [:jesup] from comment #12)
> I don't see any of the webrtc_trace logs there - did you set
> WEBRTC_TRACE_FILE=nspr ?  If not, it would have saved the log file in
> WebRTC.log somewhere (perhaps the current directory...)
> 
> It's selecting 848x480@15fps (sigh) with type 2, which should work, so we
> need the trace logs.

Yes, copied the environment variables from your comment, just replace the "whatever"

(In reply to Jan-Ivar Bruaroey [:jib] from comment #17)
> Bob, would you mind uploading a log again from one of these builds?
> http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/jbruaroey@mozilla.
> com-154962707b57/try-win32

Here's the child log, again nothing in the other one.
Flags: needinfo?(bobowen.code)
Now I'm back home I could test with the Logitech USB camera, this worked fine and there doesn't appear to be much different in the log file other than it selecting a different resolution.
Attachment #8588121 - Attachment description: log constraints used in MediaManager:5 → Part 1: log constraints used in MediaManager:5
Thanks for the logs! It shows that Hello is not adding any constraints.

I see least two bugs here:

  1) 848x480 @15fps hardly seems like the ideal pick, and
  2) Why isn't that mode working?

This patch might fix the former, because I was erroneously using the "min" frame rate of 10 as the normalizing default rather than 30, making the fitness algorithm [1] prefer lower frame-rates.

Please try again with one of these builds: 

https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/jbruaroey@mozilla.com-407df847f5c9/try-win32

Separately from this, someone might want to look at why 848x480 @15fps kVideoYUY2 isn't working (is kVideoYUY2 common? just a suspicion).

[1] http://w3c.github.io/mediacapture-main/getusermedia.html#methods-5
(In reply to Jan-Ivar Bruaroey [:jib] from comment #20)
> Created attachment 8588428 [details] [diff] [review]
> Part 2: fix: default to aPrefs.mFPS (30), not aPrefs.mMinFPS (10).
> 
> Thanks for the logs! It shows that Hello is not adding any constraints.

I'm not sure it matters, but I was using the video test at http://mozilla.github.io/webrtc-landing/gum_test.html here.
Although, I got the same problem with Hello.

> I see least two bugs here:
> 
>   1) 848x480 @15fps hardly seems like the ideal pick, and
>   2) Why isn't that mode working?
> 
> This patch might fix the former, because I was erroneously using the "min"
> frame rate of 10 as the normalizing default rather than 30, making the
> fitness algorithm [1] prefer lower frame-rates.
> 
> Please try again with one of these builds: 
> 
> https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/
> jbruaroey@mozilla.com-407df847f5c9/try-win32

This build worked, thanks ... log attached.

By the way I noticed that if I closed the browser without stopping the video first it seemed to leave the broker/chrome process running.
Does the same on Nightly and with my Logitech camera as well.
Is this a known problem?
I can reproduce this with a simple fiddle. See http://jsfiddle.net/34qxx5w1/37/

The fiddle works fine with 640x480 resolution. But as soon as trying to capture HD it does never start the video (camera LED is on and all).

I can reproduce it both in FF37 and FF40. 

I am attaching debug log from FF40. The Camera is a 720p camera which can capture 720p fine with Chrome (uvcvideo: Found UVC 1.00 device Chromebook HD WebCam (2232:1033)). You find this camera for example inside of a Chromebook Pixel.
Log file when video constraint is set to 720p but no video data is produced.
Note that depending on your camera (and how modes are selected by the constraints code), you may end up using MJPEG modes of the camera, which I've just figured out were broken: see bug 1151628 which has a patch to fix them.  I'll push a try shortly so you can download a build to verify if that causes your problem.

Note that almost all cameras support 640x480 in raw YUV which wouldn't hit this, and is our default size.  However, if the constraints code bumps you above that it might hit a MJPEG (mode 10) mode.
(In reply to Randell Jesup [:jesup] from comment #24)
> Note that depending on your camera (and how modes are selected by the
> constraints code), you may end up using MJPEG modes of the camera, which
> I've just figured out were broken: see bug 1151628 which has a patch to fix
> them.  I'll push a try shortly so you can download a build to verify if that
> causes your problem.
> 

This sounds promising i will test asap.
I just landed the patch.  Simon - you can download a build either from treeherder of inbound or the Try above, or wait for the nightly tomorrow.  This may well solve your problem

Note that Bob's problem is likely *not* MJPEG, which is mostly used at higher resolutions (mode 10, and he was selecting mode 2 which is raw YUV IIRC).  However, it doesn't hurt to try it.
Thanks. I can confirm that http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/rjesup@wgate.com-67db53f2b3eb/try-linux64/ build now works with 720p and 1080p capturing. This is the first time ever i saw HD WebRTC with Firefox which is awesome!

Now the only question remains is what released Firefox version this patch might get merged to? I have a patch to Spreed WebRTC (https://github.com/strukturag/spreed-webrtc/pull/188) bringing constraint support for Firefox and right now i have the HD constraints disabled. To make them active i need to know the minimal Firefox version required.
HD capture has been working on Mac all along, at least with internal cameras.

I hope to uplift this to Beta (38); I plan to land the disable-warnings patch, then ask for uplift at least to Aurora/39.  I really want it in 38
Attachment #8588121 - Flags: review?(rjesup)
Attachment #8588428 - Flags: review?(rjesup)
Comment on attachment 8588121 [details] [diff] [review]
Part 1: log constraints used in MediaManager:5

Review of attachment 8588121 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/media/webrtc/MediaEngineCameraVideoSource.cpp
@@ +225,5 @@
> +       aPrefs.GetWidth(), aPrefs.GetHeight(),
> +       aPrefs.mFPS, aPrefs.mMinFPS));
> +  LogConstraints(aConstraints, false);
> +  if (aConstraints.mAdvanced.WasPassed()) {
> +    LOG(("Advanced array[%d]:", aConstraints.mAdvanced.Value().Length()));

Length is typically size_t (unsigned, and may be bigger than uint).  At least make %u.  (Check GetWidth/Height/fps as well, and the logs above)
Attachment #8588121 - Flags: review?(rjesup) → review+
Attachment #8588428 - Flags: review?(rjesup) → review+
Incorporate feedback (%u, the other ones are int32_t). Carrying forward r=jesup.
Attachment #8588121 - Attachment is obsolete: true
Attachment #8589711 - Flags: review+
[Tracking Requested - why for this release]:
This will cause lower-framerate, higher-resolution modes to be selected from cameras by default (for example, 1024x576@10fps instead of 640x480@30fps).  This both cuts the framerate and may increase CPU use and bandwidth use.

Very simple, very safe fix.
Comment on attachment 8589712 [details] [diff] [review]
Part 2: fix: default to aPrefs.mFPS (30), not aPrefs.mMinFPS (10). (2) r=jesup

Approval Request Comment
[Feature/regressing bug #]: bug 1119335 I believe

[User impact if declined]: This will cause lower-framerate, higher-resolution modes to be selected from cameras by default (for example, 1024x576@10fps instead of 640x480@30fps).  This both cuts the framerate and may increase CPU use and bandwidth use.

[Describe test coverage new/current, TreeHerder]: Needs real cameras for testing currently.  In use on nightly for several weeks.

[Risks and why]: Very simple, very safe fix.  mMinFPS -> mFPS 

[String/UUID change made/needed]: none
Attachment #8589712 - Flags: approval-mozilla-beta?
Attachment #8589712 - Flags: approval-mozilla-aurora?
(In reply to Jan-Ivar Bruaroey [:jib] from comment #20)
> I see least two bugs here:
> 
>   1) 848x480 @15fps hardly seems like the ideal pick, and
>   2) Why isn't that mode working?
> 
> This patch might fix the former,

Bob, please file a new bug on 848x480 @15fps not working for you (if it's still not working for you). I'll close this now to track problem #1 and the patches requested for uplift.
Status: NEW → RESOLVED
Closed: 5 years ago
Flags: needinfo?(bobowen.code)
Resolution: --- → FIXED
(In reply to Jan-Ivar Bruaroey [:jib] from comment #36)
> to track problem #1 and the patches requested for uplift.

... the patch (singular) requested for uplift.
(In reply to Jan-Ivar Bruaroey [:jib] from comment #36)
> (In reply to Jan-Ivar Bruaroey [:jib] from comment #20)
> > I see least two bugs here:
> > 
> >   1) 848x480 @15fps hardly seems like the ideal pick, and
> >   2) Why isn't that mode working?
> > 
> > This patch might fix the former,
> 
> Bob, please file a new bug on 848x480 @15fps not working for you (if it's
> still not working for you). I'll close this now to track problem #1 and the
> patches requested for uplift.

I haven't tried it recently on my laptop, but I imagine that given that 1) is fixed I won't see 2).

Is there a way I can force it to 848x480 @15fps?
If there is, I'll try it out in the morning and file a separate bug if there's still a problem.
Flags: needinfo?(bobowen.code) → needinfo?(jib)
Try with this one on Nightly http://jsfiddle.net/jib1/6v518qyw
Flags: needinfo?(jib)
Comment on attachment 8589712 [details] [diff] [review]
Part 2: fix: default to aPrefs.mFPS (30), not aPrefs.mMinFPS (10). (2) r=jesup

Approved for uplift to aurora; this fix has been on Nightly for several weeks so has gotten some manual testing and use.
Attachment #8589712 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Florin can your team verify this fix on 39 once it's landed? Thanks!
Flags: qe-verify+
Flags: needinfo?(florin.mezei)
(In reply to Liz Henry (:lizzard) from comment #41)
> Florin can your team verify this fix on 39 once it's landed? Thanks!

As this was originally discovered on my laptop, I'll also verify once landed.
Pre-test in 39.0a2 (2015-05-08) and video failed to work as expected.

Tested in 39.0a2 (2015-05-10) and video is now working.

---------------------------------------------------

Re a possible follow-up bug ...

(In reply to Jan-Ivar Bruaroey [:jib] from comment #39)
> Try with this one on Nightly http://jsfiddle.net/jib1/6v518qyw

Tried this quickly on Friday, but couldn't get my nspr logging to work for some reason.
I've got it on my list for this week and I'll raise the follow-up bug if there is still a problem.
I'm marking this as verified based on the above comment. Thank you very much Bob for verifying this!
Flags: needinfo?(florin.mezei)
(In reply to Bob Owen (:bobowen) from comment #44)
> (In reply to Jan-Ivar Bruaroey [:jib] from comment #39)
> > Try with this one on Nightly http://jsfiddle.net/jib1/6v518qyw
> 
> Tried this quickly on Friday, but couldn't get my nspr logging to work for
> some reason.

nspr logging seems hit or miss for me with e10s. What did the fiddle itself say that it got?
Comment on attachment 8589712 [details] [diff] [review]
Part 2: fix: default to aPrefs.mFPS (30), not aPrefs.mMinFPS (10). (2) r=jesup

Approved for uplift to 38.0.5 (m-r)
Attachment #8589712 - Flags: approval-mozilla-beta? → approval-mozilla-release+
Is this something that needs consideration for esr38 as well?
Flags: needinfo?(jib)
Target Milestone: --- → mozilla40
Comment on attachment 8589712 [details] [diff] [review]
Part 2: fix: default to aPrefs.mFPS (30), not aPrefs.mMinFPS (10). (2) r=jesup

Makes sense, thanks! (note: initial request grew from beta to release).

Approval Request Comment
[Feature/regressing bug #]: bug 1119335 I believe

[User impact if declined]: This will cause lower-framerate, higher-resolution modes to be selected from cameras by default (for example, 1024x576@10fps instead of 640x480@30fps).  This both cuts the framerate and may increase CPU use and bandwidth use.

[Describe test coverage new/current, TreeHerder]: Needs real cameras for testing currently.  In use on nightly for several weeks.

[Risks and why]: Very simple, very safe fix.  mMinFPS -> mFPS 

[String/UUID change made/needed]: none
Flags: needinfo?(jib)
Attachment #8589712 - Flags: approval-mozilla-esr38?
Attachment #8589712 - Flags: approval-mozilla-esr38? → approval-mozilla-esr38+
Depends on: 1151628
Bob, could you please confirm if this bug is also fixed on ESR 38.1.0 [1]?

[1] https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.1.0esr-candidates/build1/
Flags: needinfo?(bobowen.code)
(In reply to Andrei Vaida, QA [:avaida] from comment #52)
> Bob, could you please confirm if this bug is also fixed on ESR 38.1.0 [1]?
> 
> [1]
> https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.1.0esr-
> candidates/build1/

Video is working on that build for me now.
Thanks.
Flags: needinfo?(bobowen.code)
Thanks a lot for helping out here, Bob! Updating the status of this bug as well, according to Comment 53.
Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.