391490 - JAWS reading all options for comboboxes in virtual mode, this is not usable

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[Rich Caloggero]

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a7) Gecko/2007080210 GranParadiso/3.0a7 

In Jaws virtual buffer mode, select lists are not usably read. 


Open the attached simple html page in FF and then run Jaws. 
I'm using Version 8.0.2107 


The text in the virtual buffer is as follows:
Note: text in braces represents speech generated by the screen reader when that type of object is navigated to via the virtual cursor. So, when using the arrow keys to navigate the buffer, the announcement "heading level 1" is spoken as soon as the virtual cursor moves onto the text of a level 1 heading.

The first line of the buffer is always the page title, as taken from the title element in the head.

Text of jaws virtual buffer from IE:
firefox - jaws select list bug

{heading level 1}Select List Test
{heading level 2}Single Row
{combobox}single row
one 

{Heading level 2}Multiple rows
{combobox}multiple rows
one 

End of virtual buffer contents.

Note that Jaws simply announces the select list by its title attrib (label would have been used if present), and then announced the contents of the first element. 

Jaws virtual buffer in FF:
firefox - jaws select list bug

{heading level 1}Select List Test
{heading level 2}Single Row
single row
{combobox collapsed}one
{combobox collapsed}single row
{combobox collapsed}one
{combobox collapsed}two
{combobox collapsed}three
{combobox collapsed}four

{heading level 2}multiple Rows
multiple rows
{combobox collapsed}one
{combobox collapsed}multiple rows
{combobox collapsed}one
{combobox collapsed}two
{combobox collapsed}three
{combobox collapsed}four
{combobox collapsed}five
End virtual buffer contents.

In ff, all elements are announced and the title is repeated. The word "collapsed" is spoken regardless of whether the row count is greater than 1.  This is extremely verbose and confusing.


When in "forms mode", both ff and IE act similarly (i.e. if the select list is focused and Jaws is in forms mode, you can browse the list with arrows and items are read as they are focused.  Jaws does have an annoying habbit of re-announcing the label of the select list whenever it gains focus and an arrow is pressed (this only happens on the first arrow press after focus). In IE, if you are in forms mode and tab away from and then back to a select list, then use arrows, only the items themselves are announced. The label of the select list is announced when the element gains focus, so no need to repeat it when an arrow is pressed.




Reproducible: Always

Steps to Reproduce:
1. Open the test case.
2. run Jaws if it is not already running.
3. Focus on the ff window containing the page.

Actual Results:  
See  details.

Expected Results:  
See IE results in details.

Comment 1

[Rich Caloggero]

Attachment: HTML

Comment 2

[Tim Keenan]

Since this appears to be a Jaws-only problem, this is something we would need to take up with Freedom Scientific.  Window-eyes does not exhibit this behavior.  

Comment 3

[Aaron Leventhal]

Is it also a problem in Firefox 2?

Comment 4

[Aaron Leventhal]

Attachment: Corrected testcase

The previous testcase incorrectly specified the number of rows. The correct HTML attribute is called size, not rows.

Comment 5

[Marco Zehe (:MarcoZ)]

(In reply to comment #3)
> Is it also a problem in Firefox 2?

No, Firefox 2 renders the combobox as shown for IE above, but also gives the state of "collapsed". This behaviour of showing all Combobox items is new to FF 3.

Comment 6

[Aaron Leventhal]

We may need to find out the exact day this broke. I can do that unless there is a volunteer to go through the builds at archive.mozilla.org.

Comment 7

[Marco Zehe (:MarcoZ)]

I'm on it.

Comment 8

[Marco Zehe (:MarcoZ)]

First part of the story: The first build that shows the current behaviour is from October 3rd, 2006: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20061003 Minefield/3.0a1.
However, there is a second (or rather first) part to this bug. The October 2nd build showed the first two entries of the combo box on two lines, whereas on the first line, only the first item is rendered, whereas on the second line, the first and second items are concatenated together.
I'll find out tomorrow when this started. It is definitely before the 1st September 2006.

Comment 9

[Marco Zehe (:MarcoZ)]

The last working build that did not exhibit this problem was built on June 21, 2006. The builds from June 22, 2006 through June 26, 2006, were all hosed: They did not even launch. The build made on June 27, 2006 exhibits this problem in its intermediate form: Comboboxes rendered on two lines: The first line shows the first entry, the second shows the first two entries.
So, the problem got introduced between the builds of June 21 and June 27, 2006. Sorry I can't be more specific than that!

Comment 10

[Marco Zehe (:MarcoZ)]

Turns out that this has been caused by a fix to bug 354832 - Support nsIAccessibleText for html buttons and list options. Notified Aaron, and he's on top of the issue.

Comment 11

[Aaron Leventhal]

Attachment: 1) Flatten HTML list options, but not rich list options from XUL, 2) Use correct role for ARIA listbox and option inside a combobox

This brings things exactly back to where they were before the regression. One problem with that: I notice that JAWS won't recognize a multi-select listbox at all -- it doesn't show up in the virtual buffer whatsoever. I verified this is also the case in Firefox 2.

Marco, suggestions...?

Comment 12

[Marco Zehe (:MarcoZ)]

(In reply to comment #11)
> This brings things exactly back to where they were before the regression. One
> problem with that: I notice that JAWS won't recognize a multi-select listbox at
> all -- it doesn't show up in the virtual buffer whatsoever. I verified this is
> also the case in Firefox 2.

This is definitely confirmed. The rendering changes, however, as soon as at least 1 entry is selected. By comparison: The big competitor renders multiselect listboxes even if there is no entry selected, so you can at least put the virtual cursor on it and press ENTER to invoke Forms Mode. Need a separate bug for that?

Comment 13

[Aaron Leventhal]

You mean if we make sure the first item has focus then the bug goes away? Cool.

That's bug 381602.

Comment 14

[Aaron Leventhal]

This solution would work if bug 381602 was fixed.

Comment 15

[Aaron Leventhal]

Attachment: For JAWS, make sure at least 1 item is focused/selected (but not the listbox itself)

Marco, can you try running with both this patch and the previous one, and see how well it works for you? I don't like that it says an option is selected when it isn't. Unfortunately JAWS seems to repeated text when the first item is just focused, and not selected.

Comment 16

[Aaron Leventhal]

Attachment: More detailed testcase

Comment 17

[Marco Zehe (:MarcoZ)]

(In reply to comment #15)
> Marco, can you try running with both this patch and the previous one, and see
> how well it works for you? I don't like that it says an option is selected when
> it isn't. Unfortunately JAWS seems to repeated text when the first item is just
> focused, and not selected.
 
I tried, but the first patch no longer applies, gives failed hunks. If you could update it, I'd be glad to try it.

Comment 18

[Aaron Leventhal]

Attachment: Combined patch

That's odd, since it applied fine for me, but here is a combined patch with both fixes. If you start with a fresh mozilla/accessible it should apply fine.

Comment 19

[Marco Zehe (:MarcoZ)]

> That's odd, since it applied fine for me, but here is a combined patch with
> both fixes. If you start with a fresh mozilla/accessible it should apply fine.

This patch is somehow corrupted. For one, it contains a lot of question-marked filenames at the top. After even after removing these, patch.exe crashes on the file with a "could not reserve heap memory" error and exits. Other patches apply fine.

Comment 20

[Aaron Leventhal]

The ?'s at the top don't affect it -- that just indicates files in my personal tree which are not in CVS. I'm not sure what the crash is about.

Comment 21

[Aaron Leventhal]

Attachment: Try again -- combined patch remerged to trunk

Comment 22

[Marco Zehe (:MarcoZ)]

OK, I've built with this newest patch, and have the following feedback:
1. Comboboxes now work as expected.
2. Listboxes from the "More detailed example" as well as the "Corrected testcase" don't show up at all in JAWS's virtual buffer.
3. Only if an item is actually focused by keyboard will JAWS recognize the list in question.
I did a comparison with IE, and noticed that in IE, the "editable text" nodes that represent the text outside the lists, and the list accessibles are on the same logical level. In Firefox, the lists are at the "error - unexpected variant" level, and the editable text accessibles for the text outside the lists are children of these "error - unexpected variant" nodes.
Perhaps if you made sure that they'd all be on the level as the editable text, with the actual items then being children of the list, it would work.

One more thing. When I loaded the first of the test cases titled "Corrected testcase", and then closed the tab, I always got a crash.

Comment 23

[Aaron Leventhal]

The unexpected VARIANT is because of how we expose a paragraph. Accexplore doesn't handle that.

As far as the crash, do you get it without this patch? If so, can you get a crash report or stack trace?

Comment 24

[Aaron Leventhal]

Attachment: Corrected patch -- Marco this one should work -- unfortunately the previous one was incorrect

Comment 25

[Marco Zehe (:MarcoZ)]

Report on this patch:
1. In the "Corrected testcase", both lists are now rendered. The combobox shows the selected item only, and for the listbox, the first item is rendered as being selected.
2. In the "More detailed testcase", only the very first and very last listboxes show up in virtual buffer with JAWS. The ones with disabled items, and the empty one, are all three not showing up at all.

Also, when closing the tab that contains the above sample pages, I get a crash with that build, while it works fine with a regular Nightly.

Comment 26

[Rich Caloggero]

I'm seeing slightly different behavior. I'm seeing all lists in the virtual buffer, except the empty one.
I'm seeing all select lists expanded in the virtual buffer.

The "all disabled" list shows up all expanded in the virtual buffer. When you move to it and press enter, nothing shows up in the box (i.e. you can't arrow to any of the choices). I'd suggest that this behavior be to allow one to arrow to each and then announce state as disabled or readOnly, otherwise its too 
confusing.
Similar with the second list (first item disabled); it won't let you arrow to the first item.

Multiselects are difficult. I propose the following behavior:
1. Each item should be announced, followed by whether its selected or not.
2. Simply arrowing should not change the selection of anything; currently the state of each item is never announced and the selection follows currently active element. This is confusing.
3. Toggle selection with spaceBar.

With this behavior then you could simply walk the list, and decide whether each should be selected or not, and never have to worry whether your simply perusing the list would disturb the selected state of an item.

Comment 27

[Aaron Leventhal]

Rich, I think you're looking at a current Firefox without this patch applied. Marco is testing the patch to see if it fixes the problem. He had to build his own Mozilla with the patch applied.

Comment 28

[Aaron Leventhal]

Attachment: For the moment, back to just flattening option accessibles -- I can't get it to ignore listboxes any more, in JAWS 8+

Comment 29

[Aaron Leventhal]

Attachment: Use fake focused state -- this one works, Marco tested it.

Comment 30

[Marco Zehe (:MarcoZ)]

(In reply to comment #28)
> For the moment, back to just flattening option accessibles -- I can't get it to
> ignore listboxes any more, in JAWS 8+

This one is no good. The combobox works as expected, but the lists are all not there. This is how it also is in Firefox 2.

I will now rebuild with the alternative patch.

BTW: No crashes after I cleaned everything up in my tree. Sorry about the spam on this! 

Comment 31

[Marco Zehe (:MarcoZ)]

(In reply to comment #29)
> Try this as well, it still has the fake focused state

This one gives the results we want on a clean JAWS 8 install! The single select and multi select give a "Red", which is the item in focus, but not selected. The "disabled" gives the "Green" element as the item in focus, which is what will happen if the user gets into that list anyway. The only list that doesn't show up at all is the empty listbox. But this is a corner case I think that we should live with for now.
Also, no crashes with this patch.

This is the one to go!

Comment 32

[Ginn Chen]

Comment on attachment 288520 [details] [diff] [review]
Use fake focused state -- this one works, Marco tested it.

+    if (focusedOptionNode == mDOMNode) {
+      *aState |= nsIAccessibleStates::STATE_FOCUSED; // | 
+                 //nsIAccessibleStates::STATE_SELECTED;
+    }

the comment here is not wanted, right?

search ROLE_COMBOBOX_LISTITEM
http://lxr.mozilla.org/seamonkey/search?string=ROLE_COMBOBOX_LISTITEM

I found you didn't replace all of them.
Although the missed are in comments.
These are missed,
/accessible/src/msaa/nsRoleMap.h, line 422 -- // nsIAccessibleRole::ROLE_COMBOBOX_LISTITEM
/accessible/src/atk/nsRoleMap.h, line 164 -- ATK_ROLE_MENU_ITEM, // nsIAccessibleRole::ROLE_COMBOBOX_LISTITEM 115
/accessible/src/base/nsAccessibilityService.h, line 260 -- "combobox listitem", //ROLE_COMBOBOX_LISTITEM
/accessible/src/mac/nsRoleMap.h, line 160 -- NSAccessibilityMenuItemRole, // ROLE_COMBOBOX_LISTITEM

Comment 33

[Aaron Leventhal]

Attachment: Address Ginn's comments