1951776 - Windows Voice Access does not number browser tabs

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[Nathan LaPré]

STR:

1. Launch Windows Voice Access
2. Launch Firefox and open some tabs. 10 is an effective number for demonstration.
3. With Firefox in the foreground, say "show numbers" to request that Voice Access label on-screen controls

Expected: Voice Access puts number labels over the tabs, clickable by saying "click x" where x is the number. This works in Edge.
Actual: Voice Access does not put number labels over the tabs.

Voice Access does label content and many pieces of the chrome, but not the tabs themselves. This is broken both in the IA2 -> UIA bridge and native UIA implementation. Without this functionality, users have to use the grid or some other less-convenient mechanism to click tabs.

Comment 1

[Nathan LaPré]

One other note: Windows Speech Recognition does not have this problem (it properly labels the tabs with numbers).

Comment 2

[James Teh [:Jamie] (away returning 12 Jan)]

Still reproducible. Testing shows that a focusable tab gets a number, but a non-focusable tab does not. Unfortunately, tabs other than the active tab aren't focusable in Firefox, which is probably why this breaks.

Comment 3

[James Teh [:Jamie] (away returning 12 Jan)]

One way to give inactive tabs the focusable state (without giving them a tabindex, which has other consequences) is to set aria-activedescendant="" on the tablist and role="tab" id="something" on all of the tabs. This will trigger this code. However, forcing an id when we don't really need one is kinda ugly. Also, now that we have ARIA element reflection, I wonder whether the id requirement for this check is even sensible any more.

Comment 4

[James Teh [:Jamie] (away returning 12 Jan)]

(In reply to James Teh [:Jamie] from comment #3)

Also, now that we have ARIA element reflection, I wonder whether the id requirement for this check is even sensible any more.

Filed https://github.com/w3c/aria/issues/2556 about this.

Comment 5

[James Teh [:Jamie] (away returning 12 Jan)]

Another thing to check is whether the tablist being focusable (which it isn't currently) is enough to make Voice Access happy, even if the individual tabs aren't focusable.

Comment 6

[James Teh [:Jamie] (away returning 12 Jan)]

(In reply to James Teh [:Jamie] from comment #5)

Another thing to check is whether the tablist being focusable (which it isn't currently) is enough to make Voice Access happy, even if the individual tabs aren't focusable.

Making the tablist focusable does not help. The individual tabs must be focusable in order for Voice Access to pick them up.

We did discover a bug in our UIA code, though. Even if the tabs are focusable, our ISelectionItemProvider::Select implementation calls Accessible::TakeSelection, even for a tab. That just results in the tab being focused, but not activated. We should be calling DoAction in this case, not TakeSelection.

Comment 7

[James Teh [:Jamie] (away returning 12 Jan)]

Exposing the Invoke pattern instead of the SelectionItem pattern doesn't avoid the focusable requirement. That's contrary to the UIA spec anyway, but I thought it was worth a try nevertheless.

Comment 8

[James Teh [:Jamie] (away returning 12 Jan)]

https://treeherder.mozilla.org/jobs?repo=try&landoCommitID=140032

Comment 9

[James Teh [:Jamie] (away returning 12 Jan)]

Attachment: Bug 1951776 part 1: Make ISelectionItemProvider::Select use DoAction for tabs.

Even though tabs have the selectable and selected states, Gecko doesn't support TakeSelection on them; you just click (do action) on them.
However, UIA expects that tabs should support setting selection via the SelectionItem pattern.
Tabs do not support the Invoke pattern, which is what you would use to perform the default action on most things; e.g. buttons.
Fix this by calling DoAction for tabs in our ISelectionItemProvider::Select implementation.

Comment 10

[James Teh [:Jamie] (away returning 12 Jan)]

Attachment: Bug 1951776 part 2: Always expose the focusable state on XUL tabs (unless they are disabled).

UL tabs (including those on the browser tab bar) are implemented such that only the selected tab is programmatically focusable.
Unfortunately, Windows Voice Access requires tabs to be focusable in order for them to be discovered by the "show numbers" command.
Rather than trying to change the widget to support programmatically focusing tabs (which is more difficult and risky), just always expose the focusable state for accessibility.
While this seems strange on the surface, it can be argued that unselected tabs can receive focus at some point in the future (when they're selected), even if they can't receive focus now.
There is some precedent for this: aria-activedescendant requires browsers to expose the focusable state on all possible descendants even if they aren't the current active descendant.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/0ef1fa56fb09
https://hg.mozilla.org/integration/autoland/rev/02d353e0f358
part 1: Make ISelectionItemProvider::Select use DoAction for tabs. r=morgan
https://github.com/mozilla-firefox/firefox/commit/460da25768f2
https://hg.mozilla.org/integration/autoland/rev/22dd0baab072
part 2: Always expose the focusable state on XUL tabs (unless they are disabled). r=morgan

Comment 12

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/02d353e0f358
https://hg.mozilla.org/mozilla-central/rev/22dd0baab072