1106942 - Search suggestions are read as "Unknown" in the new search UI

Status: VERIFIED FIXED

(Firefox :: Search, defect)

Description

[Marco Zehe (:MarcoZ)]

STR:
1. Run Firefox and NVDA on Windows.
2. Press Ctrl+K to go into the Search field.
3. Type in Firefox or something else.
4. Press DownArrow to go into the auto complete options.

Result: NVDA will say "Search Yahoo edit", followd by "Unknown" for every auto complete option there is.

5. Go back up to the search field and press Tab to go through the different search provider options.

Result: NVDA won't say anything.

Comment 1

[:Gijs (out for now; he/him)]

I looked at this for a bit with Florian yesterday. Considering the changes that were made here, it's highly surprising the list of suggestions broke in the way it did. Marco, I presume that this used to work with 33.1 (that is, navigating through the autocomplete/history items read their label) ? The tree body which we use for this purpose hasn't really changed - although we've added something in front of the list...

Comment 2

[Marco Zehe (:MarcoZ)]

In 33 (and 36 nightlies before the new search UI landed), the autocomplete results were accessible when pressing DownArrow. But right now, I don't see any of that tree table accessible hierarchy in the autocomplete results. I see some accessibles, but the stuff focus lands on, as you heard yourself, is definitely an unknown accessible that has no semantic info.

Comment 3

[Marco Zehe (:MarcoZ)]

Additional info: I am able to find the list of autocomplete items for the search term, but they are not where keyboard focus lands. Keyboard focus lands on some unknown child of the label "Yahoo Search", which in itself is strange.
Second, the autocomplete used to be a table, now it is announced as a list, with a list child that has the column headers, and items that are the actual autocompletes. The first one is selected, the others aren't, but again, keyboard focus is somewhere else entirely.

Comment 4

[Marco Zehe (:MarcoZ)]

Regression range confirmed, this was working fine on the 2014-11-27 build, before bug 1088660 landed. There, NVDA's developer info for the tree of search history items looked like this:

Developer info for navigator object:
name: None
role: ROLE_TABLE
states: STATE_READONLY
isFocusable: False
hasFocus: False
Python object: <NVDAObjects.IAccessible.mozilla.BrokenFocusedState object at 0x05DA3F50>
Python class mro: (<class 'NVDAObjects.IAccessible.mozilla.BrokenFocusedState'>, <class 'NVDAObjects.IAccessible.mozilla.Mozilla'>, <class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>)
description: u''
location: (1112, 93, 231, 270)
value: u'Firefox activating menu bar Linux'
appModule: <'firefox' (appName u'firefox', process ID 16028) at address 6235c10>
appModule.productName: u'Nightly'
appModule.productVersion: u'36.0a1'
TextInfo: <class 'NVDAObjects.NVDAObjectTextInfo'>
windowHandle: 2359446L
windowClassName: u'MozillaDropShadowWindowClass'
windowControlID: 0
windowStyle: -1778384896
windowThreadID: 15688
windowText: u''
displayText: u''
IAccessibleObject: <POINTER(IAccessible2) ptr=0xe837d24 at 6074350>
IAccessibleChildID: 0
IAccessible event parameters: windowHandle=2359446L, objectID=-4, childID=-382409472
IAccessible accName: None
IAccessible accRole: ROLE_SYSTEM_TABLE
IAccessible accState: STATE_SYSTEM_READONLY, STATE_SYSTEM_VALID (64)
IAccessible accDescription: u''
IAccessible accValue: u'Firefox activating menu bar Linux'
IAccessible2 windowHandle: 2359446
IAccessible2 uniqueID: -382409472
IAccessible2 role: ROLE_SYSTEM_TABLE
IAccessible2 states: IA2_STATE_VERTICAL, IA2_STATE_OPAQUE (132096)
IAccessible2 attributes: u'margin-left:0px;text-align:start;text-indent:0px;margin-right:0px;tag:tree;class:autocomplete-tree plain;margin-top:0px;margin-bottom:0px;display:-moz-box;'

Comment 5

[Marco Zehe (:MarcoZ)]

The 2014-11-28 and beyond builds show this for the tree of search history/suggestions:
Developer info for navigator object:
name: None
role: ROLE_LIST
states: STATE_READONLY
isFocusable: False
hasFocus: False
Python object: <NVDAObjects.Dynamic_ListBrokenFocusedStateMozillaIAccessible object at 0x02D44350>
Python class mro: (<class 'NVDAObjects.Dynamic_ListBrokenFocusedStateMozillaIAccessible'>, <class 'NVDAObjects.IAccessible.List'>, <class 'NVDAObjects.IAccessible.mozilla.BrokenFocusedState'>, <class 'NVDAObjects.IAccessible.mozilla.Mozilla'>, <class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>)
description: u''
location: (1058, 133, 285, 270)
value: u'Firefox Kindle Fire HDX'
appModule: <'firefox' (appName u'firefox', process ID 12928) at address 62218d0>
appModule.productName: u'Nightly'
appModule.productVersion: u'37.0a1'
TextInfo: <class 'NVDAObjects.NVDAObjectTextInfo'>
windowHandle: 722844L
windowClassName: u'MozillaDropShadowWindowClass'
windowControlID: 0
windowStyle: -1778384896
windowThreadID: 15628
windowText: u''
displayText: u''
IAccessibleObject: <POINTER(IAccessible2) ptr=0xc64436c at 6074a30>
IAccessibleChildID: 0
IAccessible event parameters: windowHandle=722844L, objectID=-4, childID=-459455344
IAccessible accName: None
IAccessible accRole: ROLE_SYSTEM_LIST
IAccessible accState: STATE_SYSTEM_READONLY, STATE_SYSTEM_VALID (64)
IAccessible accDescription: u''
IAccessible accValue: u'Firefox Kindle Fire HDX'
IAccessible2 windowHandle: 722844
IAccessible2 uniqueID: -459455344
IAccessible2 role: ROLE_SYSTEM_LIST
IAccessible2 states: IA2_STATE_VERTICAL, IA2_STATE_OPAQUE (132096)
IAccessible2 attributes: u'margin-left:0px;text-align:start;text-indent:0px;margin-right:0px;tag:tree;class:autocomplete-tree plain search-panel-tree;margin-top:0px;margin-bottom:0px;display:-moz-box;'

Comment 6

[Marco Zehe (:MarcoZ)]

In contrast, as requested by Gijs on IRC, the focused defunct item that produces the "unknown" announcement in NVDA, looks like this:
Developer info for navigator object:
name: None
role: ROLE_UNKNOWN
states: 
isFocusable: False
hasFocus: False
Python object: <NVDAObjects.IAccessible.IAccessible object at 0x06031FD0>
Python class mro: (<class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>)
description: None
location: None
value: None
appModule: <'firefox' (appName u'firefox', process ID 12928) at address 62218d0>
appModule.productName: u'Nightly'
appModule.productVersion: u'37.0a1'
TextInfo: <class 'NVDAObjects.NVDAObjectTextInfo'>
windowHandle: 722844
windowClassName: u'MozillaDropShadowWindowClass'
windowControlID: 0
windowStyle: -1778384896
windowThreadID: 15628
windowText: u''
displayText: exception: 'NoneType' object is not iterable
IAccessibleObject: <POINTER(IAccessible) ptr=0xe827a10 at 5d10580>
IAccessibleChildID: -481909152
IAccessible event parameters: windowHandle=722844, objectID=-4, childID=-481909152
IAccessible accName: exception: (-2147024809, 'The parameter is incorrect.', (None, None, None, 0, None))
IAccessible accRole: exception: (-2147024809, 'The parameter is incorrect.', (None, None, None, 0, None))
IAccessible accState: exception: (-2147024809, 'The parameter is incorrect.', (None, None, None, 0, None))
IAccessible accDescription: exception: (-2147024809, 'The parameter is incorrect.', (None, None, None, 0, None))
IAccessible accValue: exception: (-2147024809, 'The parameter is incorrect.', (None, None, None, 0, None))

The actual item that should be spoken, is found with the NVDA object navigator, and should be this:

Developer info for navigator object:
name: u'Firefox Kindle Fire HDX'
role: ROLE_LISTITEM
states: STATE_FOCUSABLE, STATE_SELECTABLE, STATE_FOCUSED, STATE_SELECTED
isFocusable: True
hasFocus: True
Python object: <NVDAObjects.IAccessible.mozilla.BrokenFocusedState object at 0x02D443D0>
Python class mro: (<class 'NVDAObjects.IAccessible.mozilla.BrokenFocusedState'>, <class 'NVDAObjects.IAccessible.mozilla.Mozilla'>, <class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>)
description: u''
location: (1058, 134, 285, 26)
value: None
appModule: <'firefox' (appName u'firefox', process ID 12928) at address 62218d0>
appModule.productName: u'Nightly'
appModule.productVersion: u'37.0a1'
TextInfo: <class 'NVDAObjects.NVDAObjectTextInfo'>
windowHandle: 722844L
windowClassName: u'MozillaDropShadowWindowClass'
windowControlID: 0
windowStyle: -1778384896
windowThreadID: 15628
windowText: u''
displayText: u''
IAccessibleObject: <POINTER(IAccessible2) ptr=0xce71a7c at 6252da0>
IAccessibleChildID: 0
IAccessible event parameters: windowHandle=722844L, objectID=-4, childID=-606777056
IAccessible accName: u'Firefox Kindle Fire HDX'
IAccessible accRole: ROLE_SYSTEM_LISTITEM
IAccessible accState: STATE_SYSTEM_SELECTABLE, STATE_SYSTEM_SELECTED, STATE_SYSTEM_FOCUSED, STATE_SYSTEM_FOCUSABLE, STATE_SYSTEM_VALID (3145734)
IAccessible accDescription: u''
IAccessible accValue: None
IAccessible2 windowHandle: 722844
IAccessible2 uniqueID: -606777056
IAccessible2 role: ROLE_SYSTEM_LISTITEM
IAccessible2 states: IA2_STATE_ACTIVE, IA2_STATE_OPAQUE (1025)
IAccessible2 attributes: u'level:1;setsize:10;posinset:1;explicit-name:true;'

Comment 7

[Florian Quèze [:florian]]

(In reply to Marco Zehe (:MarcoZ) from comment #0)
> STR:
> 1. Run Firefox and NVDA on Windows.
> 2. Press Ctrl+K to go into the Search field.
> 3. Type in Firefox or something else.
> 4. Press DownArrow to go into the auto complete options.
> 
> Result: NVDA will say "Search Yahoo edit", followd by "Unknown" for every
> auto complete option there is.

I'm rephrasing the bug summary to cover only this part.

> 5. Go back up to the search field and press Tab to go through the different
> search provider options.
> 
> Result: NVDA won't say anything.

You filed bug 1107695 to track this part.

Comment 8

[Florian Quèze [:florian]]

Attachment: Fix

So after investigating this, it turns out having a <label> in the autocomplete panel before the <tree> breaks the accessibility of the tree.

Adding role=presentation on the first <label> of the panel fixes this bug.

Another possible issue with the a11y code is that putting the role=presentation attribute on the hbox that contains the label doesn't help. From the documentation I found when googling around, I thought the role=presentation attribute was supposed to apply to the whole subtree.

Comment 9

[Florian Quèze [:florian]]

[Tracking Requested - why for this release]: This is a major accessibility regression that we shipped in 34 and the fix is simple. If we happen to make a 34.1, I would like this fix to be included.

Comment 10

[:Gijs (out for now; he/him)]

Comment on attachment 8533784 [details] [diff] [review]
Fix

Review of attachment 8533784 [details] [diff] [review]:
-----------------------------------------------------------------

r+ because the search engine name is in the search box's tooltiptext and gets exposed anyway. I don't think this particular xul:label needs to be exposed non-visually.

Comment 11

[Marco Zehe (:MarcoZ)]

role="presentation" always only applies to the element it is set on, not the sub tree. This is by design and in accordance with official ARIA 1.0 documentation. Thanks for the fix, and I agree with Gijs that this does not need to be exposed to screen readers.

Comment 12

[Florian Quèze [:florian]]

https://hg.mozilla.org/integration/fx-team/rev/4ced83ce42c9

Comment 13

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/4ced83ce42c9

Comment 14

[Florian Quèze [:florian]]

Comment on attachment 8533784 [details] [diff] [review]
Fix

Approval Request Comment
[Feature/regressing bug #]: bug 1088660
[User impact if declined]: major accessibility regression: the search suggestions are completely unaccessible to screen readers.
[Describe test coverage new/current, TBPL]: on central
[Risks and why]: very low
[String/UUID change made/needed]: none.

If we happen to ship a 34.1, I would like this patch to be part of it.

Comment 15

[Florian Quèze [:florian]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/f15b6dde1c8f

Comment 16

[Florian Quèze [:florian]]

https://hg.mozilla.org/releases/mozilla-beta/rev/b77ff1e23d57

Comment 17

[Lawrence Mandel [:lmandel] (use needinfo)]

I'm tracking for 34 to keep this on the list of ride alongs for any point release. I do not think this issue warrants a point release on its own.

Comment 18

[:Gijs (out for now; he/him)]

(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #17)
> I'm tracking for 34 to keep this on the list of ride alongs for any point
> release. I do not think this issue warrants a point release on its own.

Agreed, but could we land it on release anyway so that we don't miss it if we chemspill or something? Is that a dumb idea? :-)

Comment 19

[Petruta Horea [:phorea], Desktop Test Engineering]

Verified Firefox 35 beta 4, latest Developer Edition 36.0a2 and Nightly 37.0a1 under Win 7 64-bit.

The suggestions are now read, but in several cases it reads the same suggestion twice "<item name> <item name and the position on the list>", so if "Firefox download" is the second one on the list, it says "firefox download - firefox download two of ten level one". Opposite to this, on about:home and about:newtab the suggestions are read only once, without the specification of their position on the list.

Comment 20

[Florian Quèze [:florian]]

(In reply to Petruta Rasa [QA] [:petruta] from comment #19)

> The suggestions are now read, but in several cases it reads the same
> suggestion twice "<item name> <item name and the position on the list>", so
> if "Firefox download" is the second one on the list, it says "firefox
> download - firefox download two of ten level one". Opposite to this, on
> about:home and about:newtab the suggestions are read only once, without the
> specification of their position on the list.

I think the old UI in Firefox 34 also behaved that way.

Comment 21

[Lawrence Mandel [:lmandel] (use needinfo)]

(In reply to :Gijs Kruitbosch from comment #18)
> Agreed, but could we land it on release anyway so that we don't miss it if
> we chemspill or something? Is that a dumb idea? :-)

Not a dumb idea but I do want to limit what lands on release in case:
1. we need to spin a mobile point release that doesn't require the change in which case I don't want the extra code change in case there is any impact
2. we need to chemspill, in which case we don't want any additional code change on the branch (no ride alongs for chemspills)

Comment 22

[Petruta Horea [:phorea], Desktop Test Engineering]

(In reply to Florian Quèze [:florian] [:flo] from comment #20)
> (In reply to Petruta Rasa [QA] [:petruta] from comment #19)
> 
> > The suggestions are now read, but in several cases it reads the same
> > suggestion twice "<item name> <item name and the position on the list>", so
> > if "Firefox download" is the second one on the list, it says "firefox
> > download - firefox download two of ten level one". Opposite to this, on
> > about:home and about:newtab the suggestions are read only once, without the
> > specification of their position on the list.
> 
> I think the old UI in Firefox 34 also behaved that way.

Indeed, it's the same in the release version. I will file a new bug for this.

Marking this one as verified.

Comment 23

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 8533784 [details] [diff] [review]
Fix

We've shipped 35.0 with the fix, no need to approve this for mozilla-release.