Closed Bug 391490 Opened 13 years ago Closed 13 years ago

JAWS reading all options for comboboxes in virtual mode, this is not usable

Categories

(Core :: Disability Access APIs, defect)

x86
Windows XP
defect
Not set
normal

Tracking

()

RESOLVED FIXED

People

(Reporter: rjc, Assigned: aaronlev)

References

Details

(Keywords: access)

Attachments

(3 files, 8 obsolete files)

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a7) Gecko/2007080210 GranParadiso/3.0a7 

In Jaws virtual buffer mode, select lists are not usably read. 


Open the attached simple html page in FF and then run Jaws. 
I'm using Version 8.0.2107 


The text in the virtual buffer is as follows:
Note: text in braces represents speech generated by the screen reader when that type of object is navigated to via the virtual cursor. So, when using the arrow keys to navigate the buffer, the announcement "heading level 1" is spoken as soon as the virtual cursor moves onto the text of a level 1 heading.

The first line of the buffer is always the page title, as taken from the title element in the head.

Text of jaws virtual buffer from IE:
firefox - jaws select list bug

{heading level 1}Select List Test
{heading level 2}Single Row
{combobox}single row
one 

{Heading level 2}Multiple rows
{combobox}multiple rows
one 

End of virtual buffer contents.

Note that Jaws simply announces the select list by its title attrib (label would have been used if present), and then announced the contents of the first element. 

Jaws virtual buffer in FF:
firefox - jaws select list bug

{heading level 1}Select List Test
{heading level 2}Single Row
single row
{combobox collapsed}one
{combobox collapsed}single row
{combobox collapsed}one
{combobox collapsed}two
{combobox collapsed}three
{combobox collapsed}four

{heading level 2}multiple Rows
multiple rows
{combobox collapsed}one
{combobox collapsed}multiple rows
{combobox collapsed}one
{combobox collapsed}two
{combobox collapsed}three
{combobox collapsed}four
{combobox collapsed}five
End virtual buffer contents.

In ff, all elements are announced and the title is repeated. The word "collapsed" is spoken regardless of whether the row count is greater than 1.  This is extremely verbose and confusing.


When in "forms mode", both ff and IE act similarly (i.e. if the select list is focused and Jaws is in forms mode, you can browse the list with arrows and items are read as they are focused.  Jaws does have an annoying habbit of re-announcing the label of the select list whenever it gains focus and an arrow is pressed (this only happens on the first arrow press after focus). In IE, if you are in forms mode and tab away from and then back to a select list, then use arrows, only the items themselves are announced. The label of the select list is announced when the element gains focus, so no need to repeat it when an arrow is pressed.




Reproducible: Always

Steps to Reproduce:
1. Open the test case.
2. run Jaws if it is not already running.
3. Focus on the ff window containing the page.

Actual Results:  
See  details.

Expected Results:  
See IE results in details.
Attached file HTML (obsolete) —
Since this appears to be a Jaws-only problem, this is something we would need to take up with Freedom Scientific.  Window-eyes does not exhibit this behavior.  
Is it also a problem in Firefox 2?
Attached file Corrected testcase
The previous testcase incorrectly specified the number of rows. The correct HTML attribute is called size, not rows.
Attachment #275934 - Attachment is obsolete: true
(In reply to comment #3)
> Is it also a problem in Firefox 2?

No, Firefox 2 renders the combobox as shown for IE above, but also gives the state of "collapsed". This behaviour of showing all Combobox items is new to FF 3.
We may need to find out the exact day this broke. I can do that unless there is a volunteer to go through the builds at archive.mozilla.org.
Keywords: access
Summary: Results of Jaws screen reader announcing a select list not usable → JAWS reading all options for comboboxes in virtual mode, this is not usable
Status: UNCONFIRMED → NEW
Ever confirmed: true
I'm on it.
First part of the story: The first build that shows the current behaviour is from October 3rd, 2006: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20061003 Minefield/3.0a1.
However, there is a second (or rather first) part to this bug. The October 2nd build showed the first two entries of the combo box on two lines, whereas on the first line, only the first item is rendered, whereas on the second line, the first and second items are concatenated together.
I'll find out tomorrow when this started. It is definitely before the 1st September 2006.
The last working build that did not exhibit this problem was built on June 21, 2006. The builds from June 22, 2006 through June 26, 2006, were all hosed: They did not even launch. The build made on June 27, 2006 exhibits this problem in its intermediate form: Comboboxes rendered on two lines: The first line shows the first entry, the second shows the first two entries.
So, the problem got introduced between the builds of June 21 and June 27, 2006. Sorry I can't be more specific than that!
Turns out that this has been caused by a fix to bug 354832 - Support nsIAccessibleText for html buttons and list options. Notified Aaron, and he's on top of the issue.
This brings things exactly back to where they were before the regression. One problem with that: I notice that JAWS won't recognize a multi-select listbox at all -- it doesn't show up in the virtual buffer whatsoever. I verified this is also the case in Firefox 2.

Marco, suggestions...?
(In reply to comment #11)
> This brings things exactly back to where they were before the regression. One
> problem with that: I notice that JAWS won't recognize a multi-select listbox at
> all -- it doesn't show up in the virtual buffer whatsoever. I verified this is
> also the case in Firefox 2.

This is definitely confirmed. The rendering changes, however, as soon as at least 1 entry is selected. By comparison: The big competitor renders multiselect listboxes even if there is no entry selected, so you can at least put the virtual cursor on it and press ENTER to invoke Forms Mode. Need a separate bug for that?
You mean if we make sure the first item has focus then the bug goes away? Cool.

That's bug 381602.
This solution would work if bug 381602 was fixed.
Depends on: 381602
Blocks: fox3access
Marco, can you try running with both this patch and the previous one, and see how well it works for you? I don't like that it says an option is selected when it isn't. Unfortunately JAWS seems to repeated text when the first item is just focused, and not selected.
Attached file More detailed testcase
Attachment #288180 - Attachment is patch: false
Attachment #288180 - Attachment mime type: text/plain → text/html
(In reply to comment #15)
> Marco, can you try running with both this patch and the previous one, and see
> how well it works for you? I don't like that it says an option is selected when
> it isn't. Unfortunately JAWS seems to repeated text when the first item is just
> focused, and not selected.
 
I tried, but the first patch no longer applies, gives failed hunks. If you could update it, I'd be glad to try it.
Attached patch Combined patch (obsolete) — Splinter Review
That's odd, since it applied fine for me, but here is a combined patch with both fixes. If you start with a fresh mozilla/accessible it should apply fine.
> That's odd, since it applied fine for me, but here is a combined patch with
> both fixes. If you start with a fresh mozilla/accessible it should apply fine.

This patch is somehow corrupted. For one, it contains a lot of question-marked filenames at the top. After even after removing these, patch.exe crashes on the file with a "could not reserve heap memory" error and exits. Other patches apply fine.
The ?'s at the top don't affect it -- that just indicates files in my personal tree which are not in CVS. I'm not sure what the crash is about.
Attachment #288219 - Attachment is obsolete: true
OK, I've built with this newest patch, and have the following feedback:
1. Comboboxes now work as expected.
2. Listboxes from the "More detailed example" as well as the "Corrected testcase" don't show up at all in JAWS's virtual buffer.
3. Only if an item is actually focused by keyboard will JAWS recognize the list in question.
I did a comparison with IE, and noticed that in IE, the "editable text" nodes that represent the text outside the lists, and the list accessibles are on the same logical level. In Firefox, the lists are at the "error - unexpected variant" level, and the editable text accessibles for the text outside the lists are children of these "error - unexpected variant" nodes.
Perhaps if you made sure that they'd all be on the level as the editable text, with the actual items then being children of the list, it would work.

One more thing. When I loaded the first of the test cases titled "Corrected testcase", and then closed the tab, I always got a crash.
The unexpected VARIANT is because of how we expose a paragraph. Accexplore doesn't handle that.

As far as the crash, do you get it without this patch? If so, can you get a crash report or stack trace?
Report on this patch:
1. In the "Corrected testcase", both lists are now rendered. The combobox shows the selected item only, and for the listbox, the first item is rendered as being selected.
2. In the "More detailed testcase", only the very first and very last listboxes show up in virtual buffer with JAWS. The ones with disabled items, and the empty one, are all three not showing up at all.

Also, when closing the tab that contains the above sample pages, I get a crash with that build, while it works fine with a regular Nightly.
I'm seeing slightly different behavior. I'm seeing all lists in the virtual buffer, except the empty one.
I'm seeing all select lists expanded in the virtual buffer.

The "all disabled" list shows up all expanded in the virtual buffer. When you move to it and press enter, nothing shows up in the box (i.e. you can't arrow to any of the choices). I'd suggest that this behavior be to allow one to arrow to each and then announce state as disabled or readOnly, otherwise its too 
confusing.
Similar with the second list (first item disabled); it won't let you arrow to the first item.

Multiselects are difficult. I propose the following behavior:
1. Each item should be announced, followed by whether its selected or not.
2. Simply arrowing should not change the selection of anything; currently the state of each item is never announced and the selection follows currently active element. This is confusing.
3. Toggle selection with spaceBar.

With this behavior then you could simply walk the list, and decide whether each should be selected or not, and never have to worry whether your simply perusing the list would disturb the selected state of an item.

Rich, I think you're looking at a current Firefox without this patch applied. Marco is testing the patch to see if it fixes the problem. He had to build his own Mozilla with the patch applied.
(In reply to comment #28)
> For the moment, back to just flattening option accessibles -- I can't get it to
> ignore listboxes any more, in JAWS 8+

This one is no good. The combobox works as expected, but the lists are all not there. This is how it also is in Firefox 2.

I will now rebuild with the alternative patch.

BTW: No crashes after I cleaned everything up in my tree. Sorry about the spam on this! 

(In reply to comment #29)
> Try this as well, it still has the fake focused state

This one gives the results we want on a clean JAWS 8 install! The single select and multi select give a "Red", which is the item in focus, but not selected. The "disabled" gives the "Green" element as the item in focus, which is what will happen if the user gets into that list anyway. The only list that doesn't show up at all is the empty listbox. But this is a corner case I think that we should live with for now.
Also, no crashes with this patch.

This is the one to go!
Attachment #288518 - Attachment is obsolete: true
Attachment #288520 - Attachment description: Try this as well, it still has the fake focused state → Use fake focused state -- this one works, Marco test it.
Attachment #288520 - Flags: review?(ginn.chen)
Attachment #288373 - Attachment is obsolete: true
Comment on attachment 288520 [details] [diff] [review]
Use fake focused state -- this one works, Marco tested it.

+    if (focusedOptionNode == mDOMNode) {
+      *aState |= nsIAccessibleStates::STATE_FOCUSED; // | 
+                 //nsIAccessibleStates::STATE_SELECTED;
+    }

the comment here is not wanted, right?

search ROLE_COMBOBOX_LISTITEM
http://lxr.mozilla.org/seamonkey/search?string=ROLE_COMBOBOX_LISTITEM

I found you didn't replace all of them.
Although the missed are in comments.
These are missed,
/accessible/src/msaa/nsRoleMap.h, line 422 -- // nsIAccessibleRole::ROLE_COMBOBOX_LISTITEM
/accessible/src/atk/nsRoleMap.h, line 164 -- ATK_ROLE_MENU_ITEM, // nsIAccessibleRole::ROLE_COMBOBOX_LISTITEM 115
/accessible/src/base/nsAccessibilityService.h, line 260 -- "combobox listitem", //ROLE_COMBOBOX_LISTITEM
/accessible/src/mac/nsRoleMap.h, line 160 -- NSAccessibilityMenuItemRole, // ROLE_COMBOBOX_LISTITEM
Attachment #288520 - Flags: review?(ginn.chen)
Attachment #288520 - Attachment description: Use fake focused state -- this one works, Marco test it. → Use fake focused state -- this one works, Marco tested it.
Attachment #277397 - Attachment is obsolete: true
Attachment #288179 - Attachment is obsolete: true
Attachment #288520 - Attachment is obsolete: true
Attachment #288646 - Flags: review?(ginn.chen)
Attachment #288646 - Flags: review?(ginn.chen) → review+
Attachment #288646 - Flags: approval1.9?
Attachment #288646 - Flags: approval1.9? → approval1.9+
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Version: unspecified → Trunk
Depends on: 443764
You need to log in before you can comment on or make changes to this bug.