1609427 - Add microphone and camera to PermissionDescriptor for navigator.permissions.query

Status: ASSIGNED

(Core :: WebRTC: Audio/Video, enhancement, P2)

Description

[nikola.lukovic]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:72.0) Gecko/20100101 Firefox/72.0

Steps to reproduce:

navigator.permissions.query({ name: 'microphone' })
navigator.permissions.query({ name: 'camera' })

Actual results:

TypeError: 'name' member of PermissionDescriptor 'microphone' is not a valid value for enumeration PermissionName.

TypeError: 'name' member of PermissionDescriptor 'camera' is not a valid value for enumeration PermissionName.

Expected results:

Both requests should have returned a promise where i would be able to check the current state of permissions for both camera and microphone.
Not having an API like this implemented only deters developers to support Firefox as a platform. Also, calling it a privacy concern, in my opinion, has no basis in real world scenarios. I'm working on an app which heavily deals with real-time audio-video conferencing and other real-time interactions between users. Having an API where i can deal with permissions states would allow us to implement a smoother experience for end users and more streamlined/cleaner coding for developers.
Having to rely on hacky solutions (navigator.mediaDevices.enumerateDevices() and navigator.mediaDevices.getUserMedia()) where i don't even get events when a permission is changed is absolutely ridiculous.

Comment 1

[Seba S.]

Hi, nikola.lukovic!

Thanks for your contribution!

I don´t have a technical background to give an opinion on this so I'll add product and compoenent so we can keep on track.

Please let us know if this issue is reproduced in the latest Nightly edition. You can download it from here: https://nightly.mozilla.org/

If you still have the issue please create a new profile, you have the steps here: https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles?redirectlocale=en-US&redirectslug=Managing-profiles#w_starting-the-profile-manager
support.mozilla.orgsupport.mozilla.org

Once you have all this information, please let us know so we can continue investigating.

Regards,

Comment 2

[nikola.lukovic]

Hello,

Yes, it still shows up in the latest nightly edition.

Comment 3

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

:jib, do you have any idea about the right component for this?

Comment 4

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

The reporter is correct, we do not implement this API for camera and microphone. We have reasons for that.

Nikola, I would love to understand the use case better, if you don't mind sharing. Our current thinking is that most sites would be far better off using a cookie or localStorage to triage first-time visitors.

In short, the API in its current form is a poor fit for Firefox's getUserMedia permissions model which allows users the option to always gate camera access behind a prompt. Sites are abusing "granted" to mean "this is a returning visitor I don't need to prime" (because that works in Chrome). This pattern creates a permanent slow-lane for Firefox users who for various reasons choose to never check ☐ Remember this decision.

E.g. https://whereby.com/room1234567 after their rewrite, throws me into their "Request permission" lobby page every time, even if I check the box on their site that says I want to skip it. Blame goes to this API (or the comparable enumerateDevices() trick).

Instead, had they simply used a cookie or localStorage to record whether they've previously primed this user, everything would work fine. In Chrome, and in Firefox.

We think we understand why web sites are doing this. Chrome permanently blocks them immediately if a user says no. So priming becomes super important (and useful the first time). In Firefox there's less to fear since we don't permanently block by default.

We do have plans to eventually implement this, but we will probably end up tweaking the spec to let “prompt” mean “default treatment” and allow “granted” to potentially include prompting, except on the user's first visit (repeat getUserMedia users carry almost no blocking risk. Our telemetry bears that out).

Comment 5

[nikola.lukovic]

The thing is, we have an app that has to work predictably and to be easy to use for all levels of technical knowledge. Since we deal a lot with cameras and microphones we need a level of informational granularity to achieve something like that. The only part we're missing is to know the state of current permissions for those devices, as well as real time events about those permissions.
For example a user with no technical knowledge about such things can on accident block access to a microphone/camera and if that happens the app becomes useless/unusable. If we are to help users know what is the issue we need to have such information ready to tell them or show them what is the problem. So the biggest issue here is that the only part missing now is to know the state of permissions and to have that info ready in a form of events if they change while the app is being used.

I see now that some sites are actually abusing this feature which is unfortunate, but would a sufficient solution be for "granted" to work only in a current session? That would work for us as well. As soon as user leaves the website it gets reverted to "prompt".

Comment 6

[nikola.lukovic]

Any update on this issue?

Comment 7

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

(In reply to nikola.lukovic from comment #5)

For example a user with no technical knowledge about such things can on accident block access to a microphone/camera and if that happens the app becomes useless/unusable.

Detecting permission failure in this situation should be trivial, since getUserMedia will throw NotAllowedError. The query API is really aimed at detecting permission ahead of asking, specifically for apps that don't want to risk prompting the user. But this doesn't seem like a concern in the case you mention.

About the event: You'd probably have to interactively involve the user to unblock a permission and/or refresh the page here, correct? If it's critical for the site try again as soon as the block is cleared instead of showing the user a button to try again, then I see little cost to polling getUserMedia at some interval until it succeeds here (permissions are generally checked first, before the resource intensive parts of getUserMedia run).

I hope this is helpful, since this feature enhancement is not on our short-term radar. Thanks for filing the issue.

Comment 8

[Óvári]

Attachment: Firefox Permissions.png

Changing the Camera, Microphone and Speaker permissions with drop-downs in the address bar

To allow a peripheral, eg. camera or microphone, the X at the end need to be selected and then allowed again in the window.

Chromium has the peripherals listed and all can be changed in what seems to be a very convenient way giving a better UX. It also includes the sound (speakers/earphones).

Can Firefox’s X be changed to a drop-down for each peripheral?

What do you think?

Thank you

While the word peripheral refers to the camera, microphone and sound, these may be built-in to the device.

Comment 9

[Óvári]

Attachment: Chromium Permissions.png

Chromium seems to be a better UX with drop-downs providing better discoverability.

Comment 10

[katie]

The use case that I need this for is to implement a more user-friendly UX for non-technical users of our platform which guides them through the process of enabling and testing their camera and mic before starting their session.

As a part of this, when permission has not already been granted for mic and/or camera use, we want to tell the user that we are going to ask for permission before we ask. This message will be skipped for users that have already given permission.

There's no way for us to accomplish this without using navigator.permissions.query, and because this API isn't available for camera and microphone on firefox, that makes this more user-friendly UX impossible to implement on Firefox.

Just because you don't have the imagination to think of legitimate use cases for this API doesn't mean they don't exist, and it seems absurd to me for a browser to block access to it in an attempt to try to dictate UX decisions made by websites that need to be compatible with the browser (which is what the "reasons" described above effectively amount to).

You make the following very baseless assumption: "I suspect the main use-case for this spec is priming the user to say yes."
The service provided by the platform I am working on is virtual care (ie online doctor visits). At the point in the flow where we are asking for permission to access the camera, the user has already signed up, provided all of their details, and is waiting for a doctor to join them on a video call. We're getting a lot of complaints from non-technical users whose camera and mic aren't working, and in many cases, this is because they accidentally blocked access to the camera and/or mic to our website. This improved UX is aimed at helping to ensure that these non-technical users are able to successfully activate their camera and mic, not some nefarious plan as you seem to assume to get unsuspecting users to enable their camera for us when they might not otherwise do so.

Simply using a cookie or localstorage is honestly not really sufficient, as there are too many edge cases to worry about. It's definitely possible for a returning user to not have given permission yet, or to have revoked it at some point, for instance. It would be a lot cleaner and more reliable to simply be able to find out whether or not they've already given permission. It's absurd and insulting that Firefox prevents you from doing this just because the developers of firefox aren't imaginative enough to think of legitimate use cases for the API, and make the assumption that anyone doing so must be doing it for evil purposes.

Comment 11

[Yosef]

I couldn't agree more with Katie from #10.
If you want to try out the trick Jan-Ivar mentioned, have a look at this Stack Overflow thread which seems to be a temporary working solution for seeing the permissionStatus and if the user has a microphone/camera device. If you come up with some other workaround, do please let me know.

Comment 12

[nikola.lukovic]

Is there any update about this? It's pretty shameful that Firefox developers recommend using hacks to go around a fact that you don't want to implement a highly requested and needed feature. I currently have to call getUserMedia every second just to be sure everything works correctly. And if it fails I still have to show a vague message to users instead of being able to exactly show what's the problem. It's ridiculous to have to do something like this when other browsers have this implemented for years.

Comment 13

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Hi katie, sorry for not seeing your message sooner.

I fully understand the need to explain to new users that they're going to be asked for permission (what I call "priming"). Priming is a good thing: an opportunity for the app to explain why it needs camera and/or microphone, useful context for answering the upcoming permission question. But I hope you agree that seeing such a message every time is annoying. Let me explain below.

This message will be skipped for users that have already given permission.

I'm glad the intent is to skip the message for users that have given permission in the past, but the message will unfortunately not be skipped in browsers other than Chrome(/Edgium), since only Chrome implicitly grants persistent permission after a single use by default.

In Safari and Firefox by default, using query would prime users on every single visit. Is that what you want?

We're getting a lot of complaints from non-technical users whose camera and mic aren't working, and in many cases, this is because they accidentally blocked access to the camera and/or mic to our website.

Thanks for providing context, it's really helpful. I'm very much in favor of sites having UI to onboard non-technical users.

But are these users on Chrome? I know Chrome persistently blocks sites whose users accidentally hit "Deny" just once. Firefox does not do this.

In Firefox, hitting "Don't allow" only creates a temporary block for the current session, which is easily cleared by navigation or the user refreshing the page. Firefox users have to explicitly check the small ☐ Remember this decision box in the permission prompt, and then hit "Don't allow" to persistently block a site (in our view, if they do that then they probably mean it).

Simply using a cookie or localstorage is honestly not really sufficient, as there are too many edge cases to worry about. It's definitely possible for a returning user to not have given permission yet, or to have revoked it at some point, for instance.

localStorage-tagging users who have previously been successfully primed (meaning: they've granted permission in the past), should solve both of those problems. If not, please explain.

It would be a lot cleaner and more reliable to simply be able to find out whether or not they've already given permission.

When you say "they've already given permission", I worry you're making assumptions about how permissions work in all browsers. Do you mean "they've given permission in the past" or "they've granted my app persistent permission"?

"Having primed" and "having access" are not the same thing. They happen to be the same only in Chrome. If I understand your problem correctly, the biggest source of problems sound like first-time users who aren't expecting the camera/mic prompt, followed by Chrome-users accidentally blocking themselves. If you can explain why the localStorage solution doesn't solve this, then I'm more than willing to listen.

The query API unfortunately seems built on the assumption of a singular always-persistent permission-model, and it remains to be seen how it can be fitted to solve camera and microphone permissions in a web compatible way. It also leaks information.

Some think that sites (not yours, but we need rules for all sites) being able to turn on users' cameras and microphones at will simply because users let them once in the past, is creepy, and they prefer to know the camera cannot come on without explicit consent every time. Not everyone thinks this, that's why we give a choice.

I know it is frustrating that not all browsers work the same, but there's disagreement over which way is best, and without disagreement nothing ever gets better.

Comment 14

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Hi nikola,

I currently have to call getUserMedia every second just to be sure everything works correctly.

What does that accomplish exactly? Are you trying to detect users revoking permission? If so, you could listen to track.onended.

And if it fails I still have to show a vague message to users instead of being able to exactly show what's the problem.

What exactly is the problem?

Comment 15

[katie]

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #13)

Hi katie, sorry for not seeing your message sooner.

I fully understand the need to explain to new users that they're going to be asked for permission (what I call "priming"). Priming is a good thing: an opportunity for the app to explain why it needs camera and/or microphone, useful context for answering the upcoming permission question. But I hope you agree that seeing such a message every time is annoying. Let me explain below.

This message will be skipped for users that have already given permission.

I'm glad the intent is to skip the message for users that have given permission in the past, but the message will unfortunately not be skipped in browsers other than Chrome(/Edgium), since only Chrome implicitly grants persistent permission after a single use by default.

In Safari and Firefox by default, using query would prime users on every single visit. Is that what you want?

Yes, that is exactly what we want to do, and web browsers shouldn't be making assumptions like this about the UX of websites that run on them.
Because our website is providing virtual care, the vast majority of returning visitors will be returning months after their initial visit, so yes, we absolutely want to prime them again if they haven't granted persistent permission.

We're getting a lot of complaints from non-technical users whose camera and mic aren't working, and in many cases, this is because they accidentally blocked access to the camera and/or mic to our website.

Thanks for providing context, it's really helpful. I'm very much in favor of sites having UI to onboard non-technical users.

But are these users on Chrome? I know Chrome persistently blocks sites whose users accidentally hit "Deny" just once. Firefox does not do this.

We have users on every major browser (except IE). And yes, one of the things we need to be able to detect is if a user who accidentally hit "Deny" in Chrome. The fact that firefox doesn't do this by default doesn't change the fact that we still have to make the check

In Firefox, hitting "Don't allow" only creates a temporary block for the current session, which is easily cleared by navigation or the user refreshing the page. Firefox users have to explicitly check the small ☐ Remember this decision box in the permission prompt, and then hit "Don't allow" to persistently block a site (in our view, if they do that then they probably mean it).
That's fine. It's still possible for users to save that setting permanently, which is why we need to make the check, on every browser

Simply using a cookie or localstorage is honestly not really sufficient, as there are too many edge cases to worry about. It's definitely possible for a returning user to not have given permission yet, or to have revoked it at some point, for instance.

localStorage-tagging users who have previously been successfully primed (meaning: they've granted permission in the past), should solve both of those problems. If not, please explain.

For the reasons I already outlined. There are way too many edge cases to worry about. Maybe they are on Firefox and didn't tick "Remember this decision". Maybe they are on Firefox and did tick "Remember this decision". There are so many ways that the truth could be different from what would be suggested by whatever cookie or localstorage value we have set. Making a call to find out whether permission has already been granted is the only reliable way to know for sure if it is currently granted, and also by far the cleanest.

It would be a lot cleaner and more reliable to simply be able to find out whether or not they've already given permission.

When you say "they've already given permission", I worry you're making assumptions about how permissions work in all browsers. Do you mean "they've given permission in the past" or "they've granted my app persistent permission"?

I mean "the app has permission to use the device right now", whether that means they've given the app persistent permission in the past, or they've already given permission in the current session (the latter wouldn't be relevant in my particular case, but this is the behaviour I expect from the API based on the documentation in MDN)

"Having primed" and "having access" are not the same thing. They happen to be the same only in Chrome. If I understand your problem correctly, the biggest source of problems sound like first-time users who aren't expecting the camera/mic prompt, followed by Chrome-users accidentally blocking themselves. If you can explain why the localStorage solution doesn't solve this, then I'm more than willing to listen.

I've explained it above. If you're not capable of imagining the myriad of edge cases that can cause the localStorage value to not be reliable then I don't know what to say. The localStorage "solution" would be way too messy trying to account for all of that, and would probably still have plenty of holes where it doesn't work as intended, so it isn't really a proper solution at all. There is no way to account for and detect all of the ways in which users can change permissions after granting them without access to this API.

The query API unfortunately seems built on the assumption of a singular always-persistent permission-model, and it remains to be seen how it can be fitted to solve camera and microphone permissions in a web compatible way. It also leaks information.

No it isn't. All that the query API needs to do is tell me if I already have permission to access the device right now (per the documentation of the API on MDN: "The Permissions.query() method of the Permissions interface returns the state of a user permission on the global scope."). What is so difficult about that? It doesn't say anything about whether the permissions it is checking are persistent or not. It should not care about that. And yes, it is possible to persistently save permissions in Firefox, so on Firefox it should be checking both the persistent permissions and the temporary permissions granted.

Some think that sites (not yours, but we need rules for all sites) being able to turn on users' cameras and microphones at will simply because users let them once in the past, is creepy, and they prefer to know the camera cannot come on without explicit consent every time. Not everyone thinks this, that's why we give a choice.

That's fine, and perfectly reasonable. I'm not arguing with that. I just need to be able to use this API to determine whether or not we have permission to use the device at the time the api call is made.

I know it is frustrating that not all browsers work the same, but there's disagreement over which way is best, and without disagreement nothing ever gets better.

It's fine for Firefox to work differently from Chrome. The fact that it doesn't persist permissions by default is great. However, this is an API that is defined in MDN, is a part of the standard, and should exist on all browsers. Whether or not permissions are persisted by default in a browser should have no bearing whatsoever on the availability of this API method. An app should have a reliable way to find out if it has permission to use a device before it tries to use it.

Comment 16

[Yosef]

(In reply to Yosef from comment #11)

I couldn't agree more with Katie from #10.
If you want to try out the trick Jan-Ivar mentioned, have a look at this Stack Overflow thread which seems to be a temporary working solution for seeing the permissionStatus and if the user has a microphone/camera device. If you come up with some other workaround, do please let me know.

Comment 17

[Alberto Fabbri]

I believe that Katie is absolutely right. As a Firefox user and supporter I find it very frustrating when APIs as useful as this one are discarded without thinking twice about them.

Comment 19

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

With bug 1693677 and bug 1697284 comment 14 fixed, Firefox's permission model stands out a bit less (but still stands out):

• "Temporarily Allowed" permission is no longer revoked immediately upon track.stop() or garbage collection.
• "Temporarily Allowed" permission is now good for at least 60 minutes in the same tab (even across same-origin navigation)
• "Temporarily Allowed" remains per-device (unlike "Allowed")

Returning "granted" for the now more temporal than per-use "Temporarily Allowed" makes more sense, and onchange cannot detect garbage collection, which is one less obstacle. 👍

But I don't think this changes the equation much.

"Temporarily Allowed" permissions won't register unless developers remember to pass in a deviceId to query:

const perm = await navigator.permissions.query({name: "camera", deviceId: localStorage.lastCameraId});
console.log(perm.state); // "granted"

Developers who don't pass in deviceId are implicitly asking for permission to all cameras (including ones not yet plugged in) and will still get "prompt" always for Firefox users who never check ☐ Remember this decision:

const perm = await navigator.permissions.query({name: "camera"});
console.log(perm.state); // "prompt"

There's an inherent conflict here between keeping hot cameras behind a manual switch by default, and developers wanting to preempt prompts.

Comment 20

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

All that said, there may be some value in implementing it at this point, as it would let sites check that they're not blocked at least.

Comment 21

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

If implemented truthfully, this API becomes a fingerprinting vector, due to deviceId. I've learned that in Safari if the state is "denied" they return "prompt" until the page has called getUserMedia. This seems a useful partial mitigation we should consider.

Comment 22

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Attachment: Bug 1609427 - Add "camera" and "microphone" to permissions.query(). r?pbz

Comment 23

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Attachment: Bug 1609427 - Add ALWAYS_ASK_ACTION to nsIPermissionManager.idl. r?pbz

Depends on D183064

Comment 24

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Attachment: Bug 1609427 - Persist SitePermissions.ALWAYS_ASK for cam/mic when ☐ Remember this decision is unchecked. r?pbz

Depends on D183065

Comment 25

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Attachment: Bug 1609427 - Update test expectations for permissions.query() on camera and microphone. r?pbz

Depends on D183066

Comment 26

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Attachment: Bug 1609427 - Test permissions.query() with getUserMedia. r?pbz

Depends on D183067

Comment 27

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Note the patches here implement the behavior described in bug 1815362. An intent to ship will be forthcoming.

Comment 28

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Attachment: Bug 1609427 - Hide persisted ALWAYS_ASK permissions in the UX. r?pbz

Depends on D183068