1805582 - Add console message to indicate when MIDI access has been auto-denied due to lack of devices

Status: RESOLVED FIXED

(Core :: DOM: Device Interfaces, defect)

Description

[maccailein.mor]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0

Steps to reproduce:

With the 108 release (and the nightly 110 on 12/13/22) I was not getting the site permission add-on prompt. This was working for me in nightly 109, but now navigator.requestMIDIAccess is pending for about a second, then throws an exception. No dialog asking for approval to install the add-on appears.

This seems to fix it:

toggle midi.testing on, close browser, open browser. Now I get the site permission add-on dialog and can enumerate the fake devices
toggle midi.testing off, close browser, open browser. The fix sticks. The site permission add-on dialog still works as it did in 109 and I can enumerate real devices

Initially I thought this was some settings problem I created in testing previous versions, but I tried it on a clean Windows 11 install and I get the same problem and the workaround fixes it. I also tried this out on Ubuntu 22 (official Firefox snap) and MacOS. Both show the same behavior.

Actual results:

Site permission add-on prompt did not appear until toggling midi.testing on, closing Firefox, then trying again.

Expected results:

Site permission add-on prompt should have appeared without changing midi.testing.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox::Site Permissions' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Andrew Overholt [:overholt]]

I think this is the right component.

Comment 3

[Bobby Holley (:bholley)]

(In reply to maccailein.mor from comment #0)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0

Steps to reproduce:

With the 108 release (and the nightly 110 on 12/13/22) I was not getting the site permission add-on prompt. This was working for me in nightly 109, but now navigator.requestMIDIAccess is pending for about a second, then throws an exception. No dialog asking for approval to install the add-on appears.

Can you confirm a few things:

• Does this behavior reproduce on https://webmidi-examples.glitch.me/ ?
• Do the devices enumerate properly in Chrome?
• What appears in your console?
• Is the delay actually ~1s, or it is a randomized interval between 3s and 13s? (The latter is the expected behavior if no attached devices are found)

Comment 4

[maccailein.mor]

(In reply to Bobby Holley (:bholley) from comment #3)

Can you confirm a few things:

• Does this behavior reproduce on https://webmidi-examples.glitch.me/ ?

Yes.

• Do the devices enumerate properly in Chrome?

Yes, after plugging them in.

• What appears in your console?

Something went wrong DOMException: WebMIDI requires a site permission add-on to activate

• Is the delay actually ~1s, or it is a randomized interval between 3s and 13s? (The latter is the expected behavior if no attached devices are found)

It is more like 3s to 13s.

Based on your comments I now see that this behavior depends on whether I had MIDI devices connected when I started Firefox. The permission workflow is triggered as long as devices are plugged in before starting Firefox. If I understand correctly, this means that currently "DOMException: WebMIDI requires a site permission add-on to activate" after 3-13 seconds could be interpreted as meaning either no devices were detected or the user denied access.

You mentioned this is the expected behavior. I would have expected the permission dialog to appear, then if allowed return a MIDIAccess object with no inputs/outputs. If denied I would have guessed it would throw "SecurityError"

I am guessing this is behavior is intended to mitigate fingerprinting, while simultaneously not bothering users with the site permission prompt if they have no MIDI devices. Is that right? If that's the case, I would suggest adding the same random delay in cases where a user has chosen to "Never allow." Currently it looks like that happens in <500ms.

Or perhaps this is related to https://bugzilla.mozilla.org/show_bug.cgi?id=1437171? I'm not informed enough to be able to tell.

Comment 5

[Bobby Holley (:bholley)]

(In reply to maccailein.mor from comment #4)

Based on your comments I now see that this behavior depends on whether I had MIDI devices connected when I started Firefox. The permission workflow is triggered as long as devices are plugged in before starting Firefox.

Correct, that is expected behavior due to our lack of hotplugging support.

If I understand correctly, this means that currently "DOMException: WebMIDI requires a site permission add-on to activate" after 3-13 seconds could be interpreted as meaning either no devices were detected or the user denied access.

That's right.

You mentioned this is the expected behavior. I would have expected the permission dialog to appear, then if allowed return a MIDIAccess object with no inputs/outputs. If denied I would have guessed it would throw "SecurityError"

I am guessing this is behavior is intended to mitigate fingerprinting, while simultaneously not bothering users with the site permission prompt if they have no MIDI devices. Is that right?

Precisely. Since Chrome currently grants MIDI access to any site that requests it without any prompting (!!!), sites began using it for fingerprinting: https://mobile.twitter.com/bhology/status/1582506602266906624

If that's the case, I would suggest adding the same random delay in cases where a user has chosen to "Never allow." Currently it looks like that happens in <500ms.

If that's the case, it sounds like a bug. Can you file it and I'll take a look?

Or perhaps this is related to https://bugzilla.mozilla.org/show_bug.cgi?id=1437171? I'm not informed enough to be able to tell.

I think that's unrelated.

So to recap, I believe we can resolve this bug as worksforme, and should file another bug to look into the "Never Allow" situation. Does that sound right?

Comment 6

[Tom S [:evilpie]]

Could we log something to the web console that tells the developer that we auto-denied the request because no MIDI devices were found?

Comment 7

[Bobby Holley (:bholley)]

(In reply to Tom Schuster [:evilpie] from comment #6)

Could we log something to the web console that tells the developer that we auto-denied the request because no MIDI devices were found?

That's a nice idea, and perhaps a better way to resolve this bug. I'll do that.

Comment 8

[Bobby Holley (:bholley)]

(In reply to Bobby Holley (:bholley) from comment #5)

If that's the case, it sounds like a bug. Can you file it and I'll take a look?

I confirmed and filed this as bug 1805735, will submit a patch.

Comment 9

[maccailein.mor]

(In reply to Bobby Holley (:bholley) from comment #5)

So to recap, I believe we can resolve this bug as worksforme, and should file another bug to look into the "Never Allow" situation. Does that sound right?

Sorry I was slow to get back here after I created bug 1805734 which I think is a duplicate of your bug 1805735. I'll add a comment over there.

Thanks for the explanation and patience. In trying to figure this out I came across your tweet about the way Chrome handles this, and I completely agree.

Comment 10

[maccailein.mor]

(In reply to Bobby Holley (:bholley) from comment #8)

(In reply to Bobby Holley (:bholley) from comment #5)

If that's the case, it sounds like a bug. Can you file it and I'll take a look?

I confirmed and filed this as bug 1805735, will submit a patch.

I'm new here. Am I supposed to mark this as WORKSFORME? I don't want to overstep my authority.

Comment 11

[Bobby Holley (:bholley)]

Attachment: Bug 1805582 — Add message to console when silently denying MIDI access due to no detected devices.

Our test infrastructure runs with midi.testing enabled, which generates
virtual devices. This is mostly what we want, but it does prevent us
from testing the "no devices detected" path in automation. I've tested
it locally.

Comment 12

[Bobby Holley (:bholley)]

(In reply to maccailein.mor from comment #9)

Sorry I was slow to get back here after I created bug 1805734 which I think is a duplicate of your bug 1805735. I'll add a comment over there.

No worries at all. :-)

(In reply to maccailein.mor from comment #10)

I'm new here. Am I supposed to mark this as WORKSFORME? I don't want to overstep my authority.

I morphed this bug to track adding a console message to address the understandable confusion that led to this report. So we can keep it open, we'll take it from here. Thanks again for the report!

Comment 13

[Bobby Holley (:bholley)]

Comment on attachment 9308347 [details]
Bug 1805582 — Add message to console when silently denying MIDI access due to no detected devices.

Beta/Release Uplift Approval Request

• User impact if declined: Potential user confusion (as evidenced by this report) as to why they aren't getting prompted for MIDI access. The situation is particularly confusing because we don't support hotplugging (so the browser needs to be restarted after devices are plugged in).
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is just a single-line change to add a console message in a rare scenario, low risk.
• String changes made/needed: None (non-localized message)
• Is Android affected?: No

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4c009479afc6
— Add message to console when silently denying MIDI access due to no detected devices. r=gsvelto

Comment 15

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/4c009479afc6

Comment 16

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9308347 [details]
Bug 1805582 — Add message to console when silently denying MIDI access due to no detected devices.

Approved for 109.0b3.

Comment 17

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/2f0f0d9420d5