410807 - Auto complete popup menus not available in ATSPI

Status: RESOLVED WORKSFORME

(Core :: Disability Access APIs, defect)

Description

[Steve Lee]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b3pre) Gecko/2008010204 Minefield/3.0b3pre
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b3pre) Gecko/2008010204 Minefield/3.0b3pre

The location and search autocomplete widgets include popup menus but these are not completely exposed in ATSPI. A menu item can be seen in them but it has no child items. AFAIKS these don't even get populated when the menu is displayed.

Reproducible: Always

Steps to Reproduce:
1. use accerciser to look at either autocomplete
2. note there is a menu object but it has no children for the items
3.
Actual Results:  
no children

Expected Results:  
see children

The main menus are available at all times in the main acc object  hierarchy

Does orca handle these menus ok? if so how?

Comment 1

[Joanmarie Diggs]

> 2. note there is a menu object but it has no children for the items

I assume that's because what children get added to those objects depends on what gets typed in the entries.  Therefore what the children are and how many there are is not known (and hence cannot be exposed) at the point you are doing the examination.

> The main menus are available at all times in the main acc object  hierarchy

They are known entities that don't change -- aside from perhaps certain items' availability (i.e. greyed out or not).

> Does orca handle these menus ok? if so how?
 
When the autocomplete popup appears (either on its own or because the user has pressed Down Arrow from within the entry), an object:state-changed:showing event is issued for an object of type ROLE_WINDOW.  We then look at what we were in at the time that event occurred and based upon that make a decision about whether or not we're in such a beast or not.  If we are, we make sure we don't prevent the user from arrowing among those items (sounds silly, but since we've implemented our own caret navigation and gets lots of events we're not interested in, it's a real issue) and we make sure we present the contents of each item arrowed to. :-)

What I have found helpful is to launch Accerciser, switch to the event monitor, and then interact with the controls I'm interested in using the keyboard (i.e. as an Orca user would).  The whole time I'm simultaneously watching what gets printed out in the event monitor.  (Having a tri-head system admittedly comes in handy here ;-) ) From that I can usually figure out the circumstances (events, objects, current focus) that will enable me to identify that <whatever> is taking place and then handle it appropriately.

Since I haven't yet had time to acquaint myself with Jambu, I'm not sure if the above is sufficient info to solve your problem or not.  Hopefully so, but if not, feel free to give me a holler via email.  IMHO this is not a bug as much as it is something AT needs to be clever about. <shrugs>

Comment 2

[Steve Lee]

(In reply to comment #1)
> I assume that's because what children get added to those objects depends on
> what gets typed in the entries.  Therefore what the children are and how many
> there are is not known (and hence cannot be exposed) at the point you are doing
> the examination.

Yes I thought so (though search is pretty static) and wrote code to examine in response to the window events. I still didn't see any children of the object in the event but it turns out I was looking too early as the children get added after it is made visible (and then I had a bug). That's OK in Orca as you don't look until you get a user event for a child and obviously they're there then. In jambu we don't get that as we just show the menu and unfortunately there is no 'all children added' event so you have to either do it on time or any other event both of which give me problems.

Interestingly the autocomplete popups are a real mixed bag as Search has a menu, the location is a list and the other 2 location popups are menus but have children with various ROLES and the bookmark one is really a dialog (with strange properties).

> They are known entities that don't change -- aside from perhaps certain items'
> availability (i.e. greyed out or not).

I guess that is generally true in most app (and good UI design IMHO) but a notable exception is the file MRU. I looked on the Window event source object but didn't see the children but again that was probably the problems mentioned above. 

> When the autocomplete popup appears (either on its own or because the user has
> pressed Down Arrow from within the entry), an object:state-changed:showing
> event is issued for an object of type ROLE_WINDOW.  We then look at what we
> were in at the time that event occurred and based upon that make a decision
> about whether or not we're in such a beast or not.  

Thanks that is really helpfull info. Unfortunately in Jambu there are not the same focus driven events so behaviour is a little different. I'm driving with action (or click as work temp around) and behaviour is sometimes odd.
 
> What I have found helpful is to launch Accerciser, switch to the event monitor,
> and then interact with the controls I'm interested in using the keyboard (i.e.
> as an Orca user would).  The whole time I'm simultaneously watching what gets
> printed out in the event monitor.  (Having a tri-head system admittedly comes
> in handy here ;-) )

I'm using 2 but find the popups vanish with most operations in accerciser (like scrolling events list) so end up writing little test scripts. I have 2 heads which works OK but 3 sounds great :-)

> From that I can usually figure out the circumstances
> (events, objects, current focus) that will enable me to identify that
> <whatever> is taking place and then handle it appropriately.
 
Yes often, but I find especially with menus that the behaviour is rather non-deterimistic and I keep getting lockups. I think I still have some time dependant issues though I have tried to eliminate them.

> I'm not sure if the
> above is sufficient info to solve your problem or not.  Hopefully so, but if
> not, feel free to give me a holler via email.  

yes very thanks.

> IMHO this is not a bug as much
> as it is something AT needs to be clever about. <shrugs>

Yeah, though I'm finding the dilema is being too specific and it not working in some cases or too general and it getting in the way of other specific cases. I'm learning that developing good AT ain't just a stroll in the park ;-)  It's a slow process of learning and refining, often going over the same ground repeatedly which can be painful. I'm also hitting caching issues even though I've minimised that so need to be a bit more clever.

Thanks again for great advice.