mediaDevices.enumerateDevices() promise only resolves in foreground
Categories
(Core :: WebRTC: Audio/Video, defect)
Tracking
()
People
(Reporter: rafael, Unassigned)
References
(Depends on 1 open bug)
Details
Steps to reproduce:
I've a site that opens a websocket connection. But if I reload the window and immediately change to another window or tab, the connection isn't even attempted.
I usually debug using detached developer tools to another monitor. If I initiate the refresh from there, I need to click on the main window+tab to get it to connect.
UserAgent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/119.0
Actual results:
It seems that the WebSocket constructor waits for the tab to be active before attempting the connection, this is preventing the reconnect code from woring in background tab/window
Expected results:
(IMHO) The connection shouldn't depend on the tab/window being active. Chrome doesn't check this condition, and I couldn't find anything neither in MDN nor in whatwg.org specs.
|
||
Comment 1•1 year ago
|
||
The Bugbug bot thinks this bug should belong to the 'Core::DOM: Networking' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.
Thank you for the report, Rafael.
Do you mind providing a testcase for this. Just a simple HTML file and the steps you used to reproduce this.
This is to make sure the issue you're seeing is in the websocket code and not somewhere else.
Thanks!
|
Reporter | |
Comment 3•1 year ago
|
||
I can't reproduce this on some public websocket test sites, but I gathered some extra information (excuse me, I'd have to test that before opening a bug report): The issue doesn't appear to happen on an In Private window, but it does on a normal window with no extensions active.
Unfortunately I'm not able to post a public testcase, but I could share an URL with a valid connection token in a private channel.
Feel free to send it to necko@mozilla.com
Thanks!
|
|
Reporter | |
Comment 5•1 year ago
|
||
I've sent it, I hope you've got it. Have you had any chance to look at it?
Hi Rafael,
Does this issue only happen to you on this website, or on others too.
Looking through the sources I can see the website does things differently depending on the browser, and it also listens for visibilityChange events and reacts to those.
See the following code in Lifecycle.mjs
const getCurrentState = () => {
if (document.visibilityState === HIDDEN) {
return HIDDEN;
}
if (document.hasFocus()) {
return ACTIVE;
}
return PASSIVE;
};
Not sure if this is your code or not, but it seems like the website intentionally works like that.
I tried reproducing your steps with my own code and I couldn't get it to happen. The websocket always opens properly unless you delay it with the page visibility APIs.
Let me know if you're seeing something different please.
<!DOCTYPE html>
<html>
<head>
<title>WebSocket Playground</title>
</head>
<body></body>
<script>
console.log("loading", new Date());
setTimeout(() => {
console.log("loaded", new Date());
let WS_URL = `ws://localhost:8080/`;
const ws = new WebSocket(WS_URL);
ws.binaryType = 'arraybuffer';
ws.onopen = function() {
console.log("ws opened", new Date());
ws.send("Hi this is web client.");
};
ws.onmessage = function(e) {
console.log("Received: '" + e.data + "'");
};
}, 5000);
</script>
</html>
|
|
Reporter | |
Comment 7•1 year ago
|
||
No, lifecycle is code from google, about being active, but only used to maintain state date.
I've just found the source of the issue, I open the WebSocket at the navigator.mediaDevices.enumerateDevices() promise resolution. Promise does only resolve when asked in foreground, in Firefox. I couldn't find the point of this requirement in the standard spec
|
||
Updated•1 year ago
|
|
|
||
Comment 8•1 year ago
|
||
The severity field is not set for this bug.
:jimm, could you have a look please?
For more information, please visit BugBot documentation.
|
||
Updated•11 months ago
|
|
||
Comment 9•11 months ago
|
||
I think the promise failing to resolve is caused by this block:
if (!StaticPrefs::media_devices_unfocused_enabled()) {
// Device list changes are not exposed to unfocused contexts because the
// timing information would allow fingerprinting for content to identify
// concurrent browsing, even when pages are in different containers.
BrowsingContext* bc = window->GetBrowsingContext();
if (!bc->IsActive() || // background tab or browser window fully obscured
!bc->GetIsActiveBrowserWindow()) { // browser window without focus
return;
@jimm, I don't think there's anything Necko could do to fix this. I hope it's OK if I move it back to Audio::Video even if this is ends up WONTFIX like bug 1729889.
@Rafael, I think you can avoid this issue by setting the media.devices.unfocused.enabled pref to true in about:config.
|
|
Reporter | |
Comment 10•11 months ago
|
||
That seems the offending code, indeed. I'm not sure about that fingerprintint "problem", and I understand browsers have a great responsibility about privacy in their code, but the solution isn't valid IMHO. Excuse me streamer, as you're on a train with constant network changes/hops, you should ... and then accept DANGER, and...
My current concern now will be to detect the browser and decouple the device gathering from the websocket setup, and storing the result in sessionStorage. My code was once something like that, but (race condition) the result was that sometimes FF users couldn't stream as we didn0t know that have the capabilities.
I'd rather clarify the behaviour against the specification in the standard.
|
||
Comment 11•11 months ago
|
||
Can you clarify the purpose of the enumerateDevices() call, please?
Is it just to determine whether a recording device is present?
|
|
Reporter | |
Comment 12•11 months ago
|
||
We determine the client's capabilities as audio/video, and in the video side, if it can change camera (mobile, back/front)
|
|
||
Comment 13•11 months ago
|
||
The severity field is not set for this bug.
:jib, could you have a look please?
For more information, please visit BugBot documentation.
|
||
Comment 14•11 months ago
•
|
||
(In reply to Rafael Gawenda from comment #7)
I've just found the source of the issue, I open the WebSocket at the navigator.mediaDevices.enumerateDevices() promise resolution. Promise does only resolve when asked in foreground, in Firefox. I couldn't find the point of this requirement in the standard spec
It says: "To perform a device enumeration can proceed check, given mediaDevices, run the following steps:
- The User Agent MAY return true if device information can be exposed on mediaDevices.
- Return the result of is in view with mediaDevices.
Step 2 is a foreground check, which means Firefox is following the spec when it only resolves the promise in the foreground.
This should satisfy most applications, since device enumeration exists primarily to assist code calling getUserMedia which also requires foreground (and focus). Without more information, I'd say comment 0 seems solved by not blocking the WebSocket connection on device enumeration, as these two things seem unrelated. Please let me know if that is not a solution.
If it helps, applications that wish to detect if they'll get blocked are supposed to be able to write:
if (document.visibilityState == "visible") {
devices = await navigator.mediaDevices.enumerateDevices(); // won't block on hidden thanks to if
}
However, if you attempt this, you may run into bug 1818588. So instead of closing this as invalid, I'll mark it as a dup of that bug to up-prioritize it.
Hope that helps.
|
|
||
Comment 15•11 months ago
|
||
I've filed crbug 1513523 on Chrome. This is webkit 245864 in Safari.
|
|
Reporter | |
Comment 16•11 months ago
|
||
(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #14)
Step 2 is a foreground check, which means Firefox is following the spec when it only resolves the promise in the foreground.
It seems I didn't correctly follow the linksin the step, that was the reason I tought the focus paragraph (from my comment link) was somehow orphan.
I think you're right, and agree with the linked bugs you filled in chrome/safari. Thank you for looking into this and your time you have invested to clarify the issue..
Description
•