1699059 - [Linux Orca] The Screen Reader has an incorrect behaviour with the webRTC permission panel

Status: RESOLVED MOVED

(Firefox :: Site Permissions, defect, P2)

Description

[Andrei Purice]

Attachment: prompt.jpg.png

Affected Versions:
All latest Firefox versions

Tested on:
Ubuntu Linux

Steps to Reproduce:

1. Enable the Ubuntu screen reader before starting Firefox
2. Go to https://mozilla.github.io/webrtc-landing/gum_test.html
3. Click the Microphone and Camera test button

Expected Result:
As soon as the permission prompt is displayed, the voice-over should start reading out loud all the content from the prompt (such as with NVDA on Windows).

Actual Results:
The reader just says "Alert" and then no further reading is done for the prompt.

Note:
After refreshing the page and following again the steps the screen reader will not detect the toggled permission prompt at all, see attached screenshot.
The screen reader will fully read the prompt only if you interract with it (like if you have multiple devices connected and can interract with the dropdown). If the prompt is a label it will not be read.

Regression-Range:
Reproducible on FX 86 and beta 87 but with a slightly different outcome since only the question is detected by the screen reader: "Allow mozilla.github.io to use your camera and microphone?"

Severity:
Marking this as S2 given a11y is an important area to cover and at this point the webRTC prompt might not be functional for screen reader users. Please feel free to change the severity if need be after further investigations, I assume people who use screen readers on a daily basis will have a more technical and in-depth analysis of the severity for this issue.

Comment 1

[Paul Zühlcke [:pbz]]

I ran mozregression for the partial regression, that is the reader only saying "Alert", rather than "Allow xy to use your mic/cam"
Here is the mozregression result:

98:16.37 INFO: Last good revision: 18f24adbf3b521479f18878b5a849967ad0c6468
98:16.37 INFO: First bad revision: 5d2a519de39c5f5f419a89408bb70c1a58153c0d
98:16.37 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=18f24adbf3b521479f18878b5a849967ad0c6468&tochange=5d2a519de39c5f5f419a89408bb70c1a58153c0d

Bug 1699053 could be related.

Mike, could you take a look please?

Jamie, could you help estimating the severity of this issue? The current state where it only says "Alert" seems very problematic to me. How problematic is the older regression where it only reads the first sentence but not the rest of the doorhanger, e.g. "Allow permission.site to use your microphone?".

Comment 2

[Mike Conley (:mconley) (:⚙️)]

Hm. This might be related to bug 1699053. We tested with NVDA, but did not test with VoiceOver, and I guess aria-describedby behaves differently for those tools?

I suggest we pay attention to the result of the needinfo in bug 1699053 to see what the best way forward is, as it'll likely apply to this bug too.

Comment 3

[James Teh [:Jamie]]

Joanie, could you explain how Orca reacts to show events on an alert outside of web content? Background:

Firefox uses the alert role for non-modal notifications such as permission notifications (AKA doorhanger notifications). They don't get focus, but they should be reported to the user. You can focus them with f6, but they may also contain convenient accelerators for buttons that can be triggered without focusing them; e.g. alt+a to accept a permission.

Because some of these notifications now contain non-focusable text other than the main prompt, we started using the acc description to ensure the right text is communicated. It seems Orca doesn't pick this up, which I can understand given that the use of role alert is a bit weird here strictly speaking (though I don't think there's a better role at this stage). What's puzzling, though, is that it seems the prompt isn't read at all if there are no interactive controls other than buttons inside the alert.

If I can get some idea of what Orca expects/does here, I might hopefully be able to dream up a solution. Thanks!

Comment 4

[Joanmarie Diggs]

I don't have a camera on this machine and cannot seem to get anything to pop up on my laptop. So I just went with the microphone one.

When Orca gets a showing event for an alert it speaks the following:

• Role
• Label/name

Then it tries to figure out what the displayed information is. It proceeds in the following order:

1. Building up the string from the accessible text (implemented on the alert), expanding any embedded child text objects. If that fails,
2. Descending the accessibility tree for static text objects (role static and role label) that do NOT have a labelledby relation pointing to anything. If that fails,
3. It uses the description.

In the case of the microphone one, Orca spoke "Allow mozilla.github.io to use your microphone?" which it retrieved from the description. In fact, it spoke that description twice. So now I'll look into why that is and stop it from happening.

If I can figure out how to get the microphone+camera alert to popup on my laptop, I can see what's going on.

Orca deliberately isn't doing a full SayAll of alerts, i.e. announcing comboboxes, etc. It's waits for the user to interact with the alert -- assuming they choose to, which in this case they probably do but it's hard to programmatically know that.

Comment 5

[Joanmarie Diggs]

I fixed the double-speaking of the description in Orca master. While working on that I noticed that if the user has disabled the presentation of descriptions, Orca wasn't using it as the alert text. That is now fixed as well.

Related aside: We're sometimes getting two showing events separated out by some other events. Thus Orca is presenting the thing twice. I suppose I could add some checking in Orca, but one event is preferred. :)

And since Paul seems to think that only reading the first sentence is bad, I'll check with the Orca users to see if they want Orca to do a full SayAll of non-focused alerts. If they do, then I'll make that change in Orca. If they don't, then....

Comment 6

[Joanmarie Diggs]

I started a thread on the Orca users mailing list about how much Orca should say. The responses are an interesting read.

So far only one person has responded that it would be a good idea for Orca to read the whole thing. One person stated that they weren't sure it would be a good idea and should be an option/setting. And everyone else, so far, seems to be of the opinion best summarized here:

I would recommend something like this: "Alert. Allow mozilla.github.io to
use your microphone? Press F6 to access this alert." It is simple, easy to
understand, and doesn't make the user feel like the alert has focus.

So..... Here are my conclusions as the maintainer of Orca:

1. Unless a bunch more of the Orca user community chimes in saying they want all the verbosity the reporter is suggesting, I won't be changing Orca to read all the content like NVDA does. Sorry!!
2. Given the user feedback, I'm not sure we want Jamie to "fix" this either (e.g. putting it all someplace which will cause Orca to spew it).
3. From the quoted text above and by reading the thread, you can see that the addition of a statement about how to get into the non-focused alert would be appreciated. So if Jamie could add that to the accessible description Orca will read it.
4. It would be nice, as I mentioned before, to only have a single object:state-changed:showing event for the alert. Because there are sometimes two events, with other accessibility events interspersed, Orca is not (yet) filtering out the duplicate. And if I don't have to add such a filter, it would be great.

Lastly, I cannot reproduce Orca only saying "alert". At least in the case of the microphone only popup. I am using Orca master and Firefox Nightly. So I don't know if the issue was fixed already in either or both of our products, or if something else weird is taking place in the tester(s) environment(s).

HTH!

Comment 7

[James Teh [:Jamie]]

First, thanks so much Joanie for your speedy investigation.

(In reply to Joanmarie Diggs from comment #6)

1. Unless a bunch more of the Orca user community chimes in saying they want all the verbosity the reporter is suggesting, I won't be changing Orca to read all the content like NVDA does. Sorry!!

That's entirely reasonable. This leads me to a question, though: do you have multiple microphones on your system? If you only have one, you should just get a label for the specified microphone instead of a dropdown, and in that case, the description will include the name of that microphone (since you can't focus it and thus can't learn it about it easily otherwise).

3. From the quoted text above and by reading the thread, you can see that the addition of a statement about how to get into the non-focused alert would be appreciated. So if Jamie could add that to the accessible description Orca will read it.

Thanks; that's good feedback to have. We'll address that separately from this bug.

4. It would be nice, as I mentioned before, to only have a single object:state-changed:showing event for the alert. Because there are sometimes two events, with other accessibility events interspersed, Orca is not (yet) filtering out the duplicate. And if I don't have to add such a filter, it would be great.

Ug. I'm seeing this on Windows too. I'll investigate that separately.

Lastly, I cannot reproduce Orca only saying "alert". At least in the case of the microphone only popup.

See above re whether this is a single or multi mic system. I wonder whether things get weird for some reason when there's only a single mic.

Comment 8

[Joanmarie Diggs]

(In reply to James Teh [:Jamie] from comment #7)

That's entirely reasonable. This leads me to a question, though: do you have multiple microphones on your system? If you only have one, you should just get a label for the specified microphone instead of a dropdown, and in that case, the description will include the name of that microphone (since you can't focus it and thus can't learn it about it easily otherwise).

Apparently audio monitors are included in the list of microphones. So at the time, I had zero actual microphones plugged in. Plugging in an actual microphone/headset gives me additional choices in that alert.

Thanks; that's good feedback to have. We'll address that separately from this bug.

Awesome. Thanks!!

Ug. I'm seeing this on Windows too. I'll investigate that separately.

Thanks++

See above re whether this is a single or multi mic system. I wonder whether things get weird for some reason when there's only a single mic.

I think that would only be the case if the presence of a single mic causes none of the displayed text to be in the description.

Comment 9

[James Teh [:Jamie]]

(In reply to Joanmarie Diggs from comment #8)

See above re whether this is a single or multi mic system. I wonder whether things get weird for some reason when there's only a single mic.

I think that would only be the case if the presence of a single mic causes none of the displayed text to be in the description.

It actually adds more text to the description (the word "Microphone" and the name of the single mic). However, it does change the children that get exposed. Instead of a combo box, there'll be another label. I'm wondering whether this is somehow breaking things. Based on your description, I don't really see how, though.

Comment 10

[Timea Cernea [:tbabos][inactive]]

Hi James,
Is there enough time and a plan to fix this for uplifting in Beta 89?

Comment 11

[James Teh [:Jamie]]

We're still investigating the issue and possible fixes. It's too early to say for certain, but we're hopeful for a fix which we can uplift to beta. We will hopefully have more info in the coming few days.

Comment 12

[Pascal Chevrel:pascalc]

Eitan, are you still planning a fix for uplift in beta?

Comment 13

[Paul Zühlcke [:pbz]]

Hi Eitan, are you still working on this?

Comment 15

[Paul Zühlcke [:pbz]]

:jamie, could you help to find someone to work on this? Thanks!

Comment 16

[Eitan Isaacson [:eeejay]]

This is an Orca bug, submitted a gitlab issue and a merge request with a fix.

The description of the alert isn't used because the speech generator
picks up the "usedDescriptionForName" flag that the braille generator
set. The flag is not reset in the speech generator because the web
speech generator doesn't call the super name generator where it would
typically be reset.

Comment 17

[Paul Zühlcke [:pbz]]

Thanks for the fix!