1732410 - Wait for focus before proceeding for enumerateDevices() Promises.

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, enhancement, P1)

Description

[Karl Tomlinson (:karlt)]

The enumerateDevices() specification has changed so that it "MUST wait" for "fully active and focus" to proceed if a getUserMedia() request has not yet been granted.

If a getUserMedia() request has been granted, then "The User Agent MAY wait to proceed to the next step until document is fully active and has focus." This would provide better protection against fingerprinting and is simpler to implement given that the wait is required in some circumstances, so this is my planned approach.

Comment 1

[Karl Tomlinson (:karlt)]

"User Agents MAY add fuzzing on the timing of events to avoid cross-origin activity correlation."
Waiting for focus before dispatch would provide the best protection against cross-origin activity correlation. With the plan to wait for focus in enumerateDevices(), there would be little value in dispatching "devicechange" before enumerateDevices() could proceed.

Comment 2

[Karl Tomlinson (:karlt)]

Attachment: Bug 1732410 move enumerateDevices-after-discard-1 crashtest from Gecko to wpt r?jib

This could and perhaps should have been in wpt from the start, but there is
more benefit from the wpt harness when enumerateDevices() correctly implements
the requirement to wait for focus.

Gecko crashtests do not run with the "focusmanager.testmode" pref set and so
are easily affected by changes in OS focus. Wpt tests are run with this pref
set to emulate focus in some situations where OS focus is not available.

Some Gecko crashtests such as dom/html/crashtests/1667493.html create new
windows which can cause the OS to move focus, which does not necessarily
return after the window is closed. This can even confuse Gecko
"focusmanager.testmode" focus emulation, but wpt resets the emulated focus
between tests and so is less affected by this.

Comment 3

[Karl Tomlinson (:karlt)]

Attachment: Bug 1732410 extend the time waiting to check that enumerateDevices() Promise from a discarded document does not resolve r?jib

https://github.com/w3c/mediacapture-main/pull/574

Depends on D127044

Comment 4

[Karl Tomlinson (:karlt)]

Attachment: Bug 1732410 test enumerateDevices() with navigation r?jib

Gecko and WebKit browsers use a persisted salvageable document in history and
so the promise resolves when the document becomes active again.

Blink does not currently persist documents for bfcache, but this is planned
https://www.chromestatus.com/feature/5815270035685376
https://www.chromestatus.com/feature/6279906713403392

Depends on D127045

Comment 5

[Karl Tomlinson (:karlt)]

Attachment: Bug 1732410 test enumerateDevices() without focus r?jib

Depends on D127046

Comment 6

[Karl Tomlinson (:karlt)]

Attachment: Bug 1732410 test enumerateDevices() in background with focus in chrome r?jib

Depends on D127047

Comment 7

[Karl Tomlinson (:karlt)]

Attachment: Bug 1732410 use object instead of subject principal for chrome enumerateDevices() resistFingerprinting override r?jib,#webidl

This will simplify data structures for pending enumerateDevices() operations.

The subject principal was added for
https://bugzilla.mozilla.org/show_bug.cgi?id=1372073#c28
I don't see any current chrome callers, but chrome should be able to call this
method on a chrome document for real values. Chrome should have no need
to call this on a content document.

Depends on D127048

Comment 8

[Karl Tomlinson (:karlt)]

Attachment: Bug 1732410 remove unused support for QueryInterface to MediaDevices r?jib

Depends on D127049

Comment 9

[Karl Tomlinson (:karlt)]

Attachment: Bug 1732410 wait for focus before proceeding for enumerateDevices() Promises r?jib

https://github.com/w3c/mediacapture-main/pull/574

Focus on browser chrome widgets is accepted provided the tab is fully active
and foreground.
https://github.com/w3c/mediacapture-main/issues/752#issuecomment-742036800

Depends on D127050

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cfa820aa1978
move enumerateDevices-after-discard-1 crashtest from Gecko to wpt r=jib
https://hg.mozilla.org/integration/autoland/rev/1a2b9b3d7f59
extend the time waiting to check that enumerateDevices() Promise from a discarded document does not resolve r=jib
https://hg.mozilla.org/integration/autoland/rev/39e92d49c5ef
use object instead of subject principal for chrome enumerateDevices() resistFingerprinting override r=jib,emilio
https://hg.mozilla.org/integration/autoland/rev/805df35d47a1
remove unused support for QueryInterface to MediaDevices r=jib

Comment 11

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/31182 for changes under testing/web-platform/tests

Comment 12

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/cfa820aa1978
https://hg.mozilla.org/mozilla-central/rev/1a2b9b3d7f59
https://hg.mozilla.org/mozilla-central/rev/39e92d49c5ef
https://hg.mozilla.org/mozilla-central/rev/805df35d47a1

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/68a2bed5da24
test enumerateDevices() with navigation r=jib
https://hg.mozilla.org/integration/autoland/rev/18ada14fa683
test enumerateDevices() without focus r=jib
https://hg.mozilla.org/integration/autoland/rev/a2538bf509b9
test enumerateDevices() in background with focus in chrome r=jib
https://hg.mozilla.org/integration/autoland/rev/3e347b20ef36
wait for focus before proceeding for enumerateDevices() Promises r=jib,nika

Comment 14

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/68a2bed5da24
https://hg.mozilla.org/mozilla-central/rev/18ada14fa683
https://hg.mozilla.org/mozilla-central/rev/a2538bf509b9
https://hg.mozilla.org/mozilla-central/rev/3e347b20ef36

Comment 15

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by jgraham

Comment 16

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

I was looking for enumerateDevices-without-focus.https.html upstream and couldn't find it. This brought me here.

It looks like the patches landing in two batches here (comment 10 and comment 13) confused the WPT upstream bot to fire in comment 11, missing the patches in comment 13 entirely, causing those tests to never have been upstreamed.

We should rectify this. Also, do we have an automation problem here, or was this a snafu from the leave open flag not getting set in time?

Comment 17

[James Graham [:jgraham]]

https://searchfox.org/mozilla-central/source/testing/web-platform/mozilla/tests/mediacapture-streams/enumerateDevices-without-focus.https.html is under testing/web-platform/mozilla/tests and so doesn't get synced. testing/web-platform/tests/mediacapture-streams/enumerateDevices-with-navigation.https.html did get synced, so I think this worked as intended from a sync point of view.