Closed Bug 1420585 Opened 2 years ago Closed 2 years ago

getUserMedia hangs when constraints can't be met

Categories

(Core :: WebRTC: Audio/Video, defect, P3)

57 Branch
defect

Tracking

()

RESOLVED FIXED
mozilla59
Tracking Status
firefox59 --- fixed

People

(Reporter: john357smith, Assigned: mchiang, NeedInfo)

Details

(Whiteboard: [need info mchiang 2017-12-12])

Attachments

(17 files)

1.69 KB, text/html
Details
1.68 KB, text/html
Details
5.99 MB, video/mp4
Details
5.69 MB, video/mp4
Details
1.03 MB, application/zip
Details
1.64 MB, application/zip
Details
956.95 KB, application/zip
Details
1.07 MB, application/zip
Details
1.04 MB, application/zip
Details
1.56 MB, application/zip
Details
485.68 KB, image/png
Details
1.42 MB, application/zip
Details
1010.10 KB, application/zip
Details
1.02 MB, application/zip
Details
1.06 MB, application/zip
Details
59 bytes, text/x-review-board-request
jib
: review+
Details
59 bytes, text/x-review-board-request
jib
: review+
Details
Attached file index.html
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.102 Safari/537.36 Vivaldi/1.93.955.48

Steps to reproduce:

Set constraints to low max width/height and repeat call of navigator.mediaDevices.getUserMedia() with higher width/height in case an error Promise is returned. 

Code example is attached.

Tested with success: desktop Firefox, tablet iGET G71, desktop Chrome and Chrome Android
Tested with no success: Samsung S6 Edge, Huawei P8, Samsung Galaxy Nexus



Actual results:

An error Promise is returned for the first time but repeated call with different constraints just hangs and no Promise is returned even with page reloaded. You have to quit Firefox and start again to get same behaviour.



Expected results:

An error Promise is returned for each unsuccessful call until constraints are satisfied in which case a success Promise is returned.
It has been almost 2 weeks and no response ... will somebody look at this problem?

Thanks.
Munro, 
Can you help check it?
Flags: needinfo?(mchiang)
This is because you set a constraint which limits max resolution size to 160x120.

width: {
  max: 160
},
height: {
  max: 120
}

Enlarge the resolution or change max to ideal could fix this problem.
Component: Audio/Video → WebRTC: Audio/Video
Flags: needinfo?(mchiang)
Product: Firefox for Android → Core
Version: Firefox 57 → 57 Branch
I'm aware how constraints work but that's not what this is problem about - if you set very low resolution size which can't be met a navigator.mediaDevices.getUserMedia() should return an error promise. But it doesn't - instead it hangs infinitely without returning anything. This happens under one condition - it's a repeated call of function. Use case is described in description of this bug and attached is a code where you can test it.

Example:

1. set constraints to some really very low values - let's say 10 and 10.
2. call navigator.mediaDevices.getUserMedia() with these constraints
3. an error promise should be returned (unless your cam is able to handle such a resolution which most probably doesn't ...)
4. change constraints to 20 and 20 and call it again
5. navigator.mediaDevices.getUserMedia()never returns anything - just hangs until you restart browser (page reload doesn't help)

Doesn't work on all my phones (see above), works with one tablet ...
Attached file index.html
This attachment works well on my Pixel device.
I got numerous Overconstraint error.

May need to borrow a Samsung device.
Friendly reminder to go back on it when you get a chance.
Flags: needinfo?(mchiang)
Whiteboard: [need info mchiang 2017-12-12]
(In reply to Munro Mengjue Chiang [:mchiang] from comment #5)
> Created attachment 8936416 [details]
> index.html
> 
> This attachment works well on my Pixel device.
> I got numerous Overconstraint error.
> 
> May need to borrow a Samsung device.

Yes with the first run I also got a lot of Overconstraint errors but after the page reload just got "Testing 10" and nothing happened. But there is one weird behaviour - if all constraints will be invalid (could not be met) you can repeat it as many times as you want. But once there is success met and you get permission dialog to access your cam you never get a success promise nor your can't repeat it.
I will try to find a Samsung device and reproduce it today.
Flags: needinfo?(mchiang)
Cannot repro this bug with Samsung S7.
Hmm, that's weird ... Did you end up with permission dialog to cam (success)? And after page refresh you were still able to see it again? Is there any chance you could test it on another device?
(In reply to john357smith from comment #10)
> Hmm, that's weird ... Did you end up with permission dialog to cam
> (success)? And after page refresh you were still able to see it again?
Yes, please see the attached video.
Could you also record a clip showing how you repro this bug?

> Is there any chance you could test it on another device?
I have test Google Pixel / Samsung S7 / Samsung S8 edge.
All of them works fine. The models you listed are quite old that I cannot find any of them.
I put more log and build a test build (https://queue.taskcluster.net/v1/task/Z1EFZIBPTlm_RthjdEAn2g/runs/0/artifacts/public/build/target.apk).
Could you help me run attachment 8936416 [details] and collect adb logcat log?
There should be some "Munro Test" pattern in it.
Flags: needinfo?(john357smith)
Thanks for your patience. I installed Firefox Nightly as you proposed above, run attachment and record the screen (attached) - currently only for Samsung S6 Edge (as a newest one). Let me know if you need the same procedure also for other devices.
Flags: needinfo?(john357smith)
Thanks for the video.
Could you also provide the adb logcat log?
Flags: needinfo?(john357smith)
Attached file logcat.zip
Flags: needinfo?(john357smith)
ADB logcat attached.
Thanks for the log.
It looks like gUM was stuck and never return.
Could you help me install the below test build and collect logs again?
https://queue.taskcluster.net/v1/task/OSNP06F4TEStO9jjCfEGeA/runs/0/artifacts/public/build/target.apk

Please also provide the bugreport (adb bugreport).
Flags: needinfo?(john357smith)
Attached file bugreport_20171221.zip
Flags: needinfo?(john357smith)
Attached file logcat_20171221.zip
New build installed, tested, bugreport and logcat attached.
We are very close.
From the log, when we request webrtcUI shows the permission dialog, it doesn't show it.
And we are waiting for the user's approval or denial.
Please help me confirm this with the final test build, then I will loop in front-end team to help check.
No bugreport needed.

https://queue.taskcluster.net/v1/task/D84AB_ZGTMuBZY2xS1Medw/runs/0/artifacts/public/build/target.apk
Flags: needinfo?(john357smith)
Attached file logcat_20171221_2.zip
Flags: needinfo?(john357smith)
Sure, no problem. New build installed, tested and just to be sure a new logcat attached. Thanks.
Looks like MediaManager thread is stuck.
Please help collect more log with
https://queue.taskcluster.net/v1/task/beH8TxKvSaaMxnlrzcC_xQ/runs/0/artifacts/public/build/target.apk
Flags: needinfo?(john357smith)
Attached file logcat_20171230.zip
Flags: needinfo?(john357smith)
New build installed, tested a logcat attached. Thanks.
May I know which rom you flash to your Samsung S6 edge device?
According to the log, it seems this device is running Android Oreo, but I don't find any official announcement that this device has been updated to Oreo officially.
Flags: needinfo?(john357smith)
I don't know what do you mean but I didn't flash my phone or did anything like that. I bought it sometime in April/May this year (as a new one) and let regular updates to keep going. Nothing else ... Should I try my second device Huwaei P8?

Thanks.
Flags: needinfo?(john357smith)
BTW - I looked into the "SW info" in my phone and it's claiming running on Android 7.0 (Nougat). 

According to these articles it's possible to get 8.0 for S6 but probably it will take some time:

http://www.rootupdate.com/2017/03/names-of-samsung-devices-updating-to-android-8-0-oreo.html
http://www.trustedreviews.com/news/samsung-galaxy-s6-android-oreo-update-3343072
Thanks for your information.

Could you help

1) collect log with the apk (running attachment 8936416 [details])
https://queue.taskcluster.net/v1/task/Pl3SmQvRQFy4ly1x0Jd9Gg/runs/0/artifacts/public/build/target.apk

2) use the apk to browse the website https://mozilla.github.io/webrtc-landing/gum_test.html ; press "Video", grant the permission and check if you can see camera preview.

3) Install the app "Open Camera" in Google Play and see if you can see camera preview while running this app.
Flags: needinfo?(john357smith)
Attached file logcat_20171231.zip
Flags: needinfo?(john357smith)
1) app installed but crashed first time I started it. After restart it started OK (result same as with previous versions)
2) https://mozilla.github.io/webrtc-landing/gum_test.html - without FF restarting no cam permission but after FF restart it showed permission dialog (same behaviour as with testing of 8936416 attachment)
3) app installed - when this app is running gum_test.html is not working (either black screen or just freezed picture when I start "Open Camera" after). Same behaviour with opposite order.

logcat attached from whole sessions of testing above.

Thanks.
2) For the gum_test case, after you see the permission dialog and grant the permission, can you see camera preview like the attachment?
Flags: needinfo?(john357smith)
Yes I was able to see cam preview (after I approved it in permission dialog) till I started "Open Camera" app - then picture freezed. But - permission dialog was shown after I restarted FF (same problem as with point 1).
Flags: needinfo?(john357smith)
Attached file logcat_20180101.zip
Flags: needinfo?(john357smith)
App installed and tested - now it behaves differently. It doesn't hang (after reload) but asking me for permission twice (or even three times) despite I allowed it (clicked on "share").

Logcat attached.

Thanks.
Assignee: nobody → mchiang
Please help use chrome on your S6 edge to browse this web page.
https://jsfiddle.net/z2vpambc/

Please collect logs and also let me know if you can see the camera preview.
Flags: needinfo?(john357smith)
Flags: needinfo?(john357smith)
Log attached. Web loaded in Chrome:

- Enumerate!: output of devices was shown
- gUM!: preview of my front cam was shown

Thanks.
(In reply to john357smith from comment #42)
> Log attached. Web loaded in Chrome:
> 
> - Enumerate!: output of devices was shown
> - gUM!: preview of my front cam was shown
> 
> Thanks.
Sorry, I want to test back camera.
Please help test again with
https://jsfiddle.net/munrochiang/z2vpambc/6/
Flags: needinfo?(john357smith)
Flags: needinfo?(john357smith)
Test repeated, result is same - list of devices and back camera was shown.

Thanks.
Attached file logcat_20180104.zip
Flags: needinfo?(john357smith)
App installed and started, behaviour is same as with official FF release - hangs after page reload. Logcat attached.
This should be Samsung S6 Edge android.hardware.camera interface bug.
It doesn't support the resolution 176x144 although it claims it does.
For long term, Firefox should use android.hardware.camera2 since android.hardware.camera was deprecated.
For now, I will implement a patch which would reject the promise when we encounter this problem.
Thanks. After you make a patch can you send me an apk so I can test it on other devices where this problem also occurred (P8 amd Nexus)?
The critical part of the patch has been contained in Comment 37.
You can use it to test first.
(In reply to john357smith from comment #39)
> App installed and tested - now it behaves differently. It doesn't hang
> (after reload) but asking me for permission twice (or even three times)
> despite I allowed it (clicked on "share").
> 
> Logcat attached.
> 
> Thanks.

You are asked for permission multiple times because We keep encountering the 176x144 bug and reject the promise, and your code keep trying with larger resolution.
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Priority: -- → P3
Comment on attachment 8940084 [details]
Bug 1420585 - remove unnecessary stopCapture which causes deadlock.

https://reviewboard.mozilla.org/r/210364/#review216114

::: media/webrtc/trunk/webrtc/modules/video_capture/android/java/src/org/webrtc/videoengine/VideoCaptureAndroid.java
(Diff revision 1)
> -    if (camera != null) {
> -      Exchanger<Boolean> resultDropper = new Exchanger<Boolean>();
> -      stopCaptureOnCameraThread(resultDropper);
> -      exchange(resultDropper, false);
> -    }

For some specific device, e.g., Samsung s6 edge, camera.setParameters(parameters) would throw an exception when the resolution is set to 176 x 144. and then this snippet will be executed.

stopCaptureOnCameraThread() is supposed to be called by MediaManager thread like we did at [1]. It uses Exchanger [2] to synchronize and swap datas with MediaManager thread. Calling this function directly in CameraThread, like what this snippet does, will make it block at [3].

There is no need to call stopCapture here. The state will still be correct. So we remove this snippet.

[1] https://searchfox.org/mozilla-central/rev/652fbd6270de0d3ec424d2b88f8375ff546f949f/media/webrtc/trunk/webrtc/modules/video_capture/android/java/src/org/webrtc/videoengine/VideoCaptureAndroid.java#267

[2] https://developer.android.com/reference/java/util/concurrent/Exchanger.html

[3] https://searchfox.org/mozilla-central/rev/652fbd6270de0d3ec424d2b88f8375ff546f949f/media/webrtc/trunk/webrtc/modules/video_capture/android/java/src/org/webrtc/videoengine/VideoCaptureAndroid.java#309
s/will make it block at [3]./will make it block at [3] and also block MediaManager thread.
Comment on attachment 8940084 [details]
Bug 1420585 - remove unnecessary stopCapture which causes deadlock.

https://reviewboard.mozilla.org/r/210364/#review216256

::: media/webrtc/trunk/webrtc/modules/video_capture/android/java/src/org/webrtc/videoengine/VideoCaptureAndroid.java
(Diff revision 1)
> -    if (camera != null) {
> -      Exchanger<Boolean> resultDropper = new Exchanger<Boolean>();
> -      stopCaptureOnCameraThread(resultDropper);
> -      exchange(resultDropper, false);
> -    }

Maybe leave a code comment here about this?
Attachment #8940084 - Flags: review?(jib) → review+
Comment on attachment 8940085 [details]
Bug 1420585 - refine logs.

https://reviewboard.mozilla.org/r/210366/#review216282

::: dom/media/webrtc/MediaEngineRemoteVideoSource.cpp:301
(Diff revision 1)
> +      LOG(("ChooseCapability(kFitness) for mTargetCapability and mCapability ++"));
>        if (!ChooseCapability(aNetConstraints, aPrefs, aDeviceId, mCapability, kFitness)) {

Log message says "for mTargetCapability and mCapability" but arg is only mCapability... can you explain?
Attachment #8940085 - Flags: review?(jib) → review+
Comment on attachment 8940085 [details]
Bug 1420585 - refine logs.

https://reviewboard.mozilla.org/r/210366/#review218436

::: dom/media/webrtc/MediaEngineRemoteVideoSource.cpp:301
(Diff revision 1)
> +      LOG(("ChooseCapability(kFitness) for mTargetCapability and mCapability ++"));
>        if (!ChooseCapability(aNetConstraints, aPrefs, aDeviceId, mCapability, kFitness)) {

Please see my below comment.

::: dom/media/webrtc/MediaEngineRemoteVideoSource.cpp:307
(Diff revision 1)
>        if (!ChooseCapability(aNetConstraints, aPrefs, aDeviceId, mCapability, kFitness)) {
>          *aOutBadConstraint = FindBadConstraint(aNetConstraints, *this, aDeviceId);
>          return NS_ERROR_FAILURE;
>        }
> +      LOG(("ChooseCapability(kFitness) for mTargetCapability and mCapability --"));
>        mTargetCapability = mCapability;

Because we assign mCapability to mTargetCapability here.
Pushed by mchiang@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f5b452454285
remove unnecessary stopCapture which causes deadlock. r=jib
https://hg.mozilla.org/integration/autoland/rev/0e8dc361a38f
refine logs. r=jib
https://hg.mozilla.org/mozilla-central/rev/f5b452454285
https://hg.mozilla.org/mozilla-central/rev/0e8dc361a38f
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
You can also take a look at this tool from the samsung. You can upgrade or flash your firmware as well as install Custom ROM's using the odin tool by samsung. Here is the link to download: https://www.androidstrike.com/download-odin/
unable to access http://beststockbrokerindia.com through mozilla firefox but is accessible through internet explorer ..why?
you can check samsung galaxy s9 plus full feature and check all the details of samsung galaxy s9 plus

https://www.latesttechmedia.in/2018/08/samsung-galaxy-s9.html
Flags: needinfo?(udhbhav.tanwer05)
Yes with the first run I also got a lot of Overconstraint errors but after the page reload just got "Testing 10" and nothing happened. But there is one weird behaviour - if all constraints will be invalid (could not be met) you can repeat it as many times as you want. But once there is success met and you get permission dialog to access your cam you never get a success promise nor your can't repeat it.Gujarati Movie Download
https://broadposts.com/download-gujarati-movies/
Flags: needinfo?(broadposts)

Yes with the first run I also got a lot of Overconstraint errors but after the page reload just got "Testing 10" and nothing happened.

https://allthetechnews.yolasite.com

https://mirrorsitelist.jimdosite.com/

https://indianmatrimonylist.weebly.com

https://whatisipaddress.webs.com/

Get More Information About it on GeekTrickz

This is because you set a constraint which limits max resolution size to 160x120.
width: max: 160
height: max: 120
Enlarge the resolution or change max to ideal could fix this problem.

https://andnewsindia.com/ddr3-vs-ddr4/

You need to log in before you can comment on or make changes to this bug.