245367 - Expose tree view position in tree item accessibles

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect, P1)

Description

[Aaron Leventhal]

The MSAA API doesn't have a good way of exposing the "n of m" positional
information of a tree item within it's siblings. This information is important
because screen readers provide that as users navigate around the tree. For
example, in the following case the screen reader would say "Tree item Oranges,
Level 3, 2 of 4"

Food
  Fruit
     Apples
      ** Oranges **
      Grapes
      Bananas

This kind of information is sometimes exposed in get_accDescription().

As in description = "L3, 2 of 4"

The screen reader can parse the information out of the string for tree items.

I'm not sure if ATK has a way of dealing with this. Since technically the tree
item really could have a description, Sun will probably want to think of another
way of exposing this information. Kyle, any ideas?

Comment 1

[Aaron Leventhal]

Bill, maybe you can answer for ATK on this one?

Comment 2

[Bill Haneman]

ATK uses explicit indices for sibling relations, i.e. instead of "next-child"
and "prev-child" we say "getChildAtIndex (i)".

We don't explicitly pass level info in ATK, so this must be determined by
context, i.e. the "level" is determined from how many levels of containership
there are between the outermost "tree" or "table" container and the descendant.

Comment 3

[Aaron Leventhal]

Bill, so does the sibling number always get reported in events?

Comment 4

[Bill Haneman]

In events from objects, we only report a reference to the object (and small
amounts of event-specific info).  If we need sibling-number info (and haven't
cached info about this object), we can call
atk_object_get_index_in_parent (object);

Comment 5

[Aaron Leventhal]

That's a good way of doing it.

On the other hand, we better make sure that neither getChildAtIndex(i) or
atk_object_get_index_in_parent(object) are O(n)

I think a lot of architectures would take the lazy way out for those. Mozilla
might be -- we better look.

Comment 6

[Aaron Leventhal]

(In reply to comment #4)
> In events from objects, we only report a reference to the object (and small
> amounts of event-specific info).  If we need sibling-number info (and haven't
> cached info about this object), we can call
> atk_object_get_index_in_parent (object);
> 
> 

Hmm, atk_object_get_index_in_parent won't work because the visual tree hierarchy
doesn't match the accessibility hierarchy.

Food
  Fruit
     Apples
      ** Oranges **
      Grapes
      Bananas

This is exposed as:
tree object
   column heading list
      colhead1
      colhead2
      etc.
   food
   fruit
   apples
   oranges
   grapes
   bananas

The children are currently all exposed at the same level. We could fix this, but
then there is the issue of the column heading list being in there. The AT could
subtract that out.

Another issue comes when there is a tree view with table cells, such as in mail.
Currently every cell is a direct child of the tree, so index_in_parent would
give a completely unexpected answer.

Even tree views with table cells can have hierarchy levels -- e.g. the thread
view in mail.

Finally, I think we should be exposing tree views with no hierarchy levels as
lists. In my experience, the user doesn't view those as tree views.

Comment 7

[Aaron Leventhal]

Attachment: 1) Implement GetDesciption for MSAA xul tree items 2) Make XUL tree item cache work outisde of ATK

Comment 8

[Louie Zhao]

Comment on attachment 151627 [details] [diff] [review]
1) Implement GetDesciption for MSAA xul tree items 2) Make XUL tree item cache work outisde of  ATK

This patch will break Linux build because nsXULTreeAccessibleWrap.h is not
exported. accessible/src/xul/nsXULTreeAccessible.h can't find it. I am not sure
whether Mac or Other  still has this problem.

Comment 9

[Aaron Leventhal]

Attachment: Make it not break builds on non-windows platforms

Comment 10

[Aaron Leventhal]

Comment on attachment 151661 [details] [diff] [review]
Make it not break builds on non-windows platforms

Louie is that better?

Comment 11

[Louie Zhao]

Comment on attachment 151661 [details] [diff] [review]
Make it not break builds on non-windows platforms

still can't work for Linux build because nsXULTreeAccessibleWrap.h needs
nsXULTreeAccessible.h  .
I try to add 
 LOCAL_INCLUDES = \
+  -I$(srcdir) \
   -I$(srcdir)/../base \

   -I$(srcdir)/../html \
in Makefile under accessible/src/xul and building is ok.

I have a question about crossplatform issue.

> 
> #ifdef MOZ_ACCESSIBILITY_ATK
>@@ -383,7 +387,7 @@
>   GetCacheEntry(*mAccessNodeCache, (void*)(aRow * kMaxTreeColumns + columnIndex), getter_AddRefs(accessNode));
>   if (!accessNode)
>   {
>-    accessNode = new nsXULTreeitemAccessible(this, mDOMNode, mWeakShell, aRow, col);
>+    accessNode = new nsXULTreeitemAccessibleWrap(this, mDOMNode, mWeakShell, aRow, col);
>     if (! accessNode)
>       return NS_ERROR_OUT_OF_MEMORY;
>     PutCacheEntry(*mAccessNodeCache, (void*)(aRow * kMaxTreeColumns + columnIndex), accessNode);
>@@ -572,9 +576,9 @@
>   }

Since these codes are crossplatform, nsXULTreeitemAccessibleWrap is created
instead of nsXULTreeitemAccessible on Mac. But nsXULTreeitemAccessibleWrap.h is
only included on Windows and Unix. I suppose build will break on Mac.

Comment 12

[Aaron Leventhal]

Attachment: Better patch

Sorry I forgot to include my changes to the Makefiles for mac and other. They
include a typedef for nsAccessibleTreeitemWrap.

Thanks for doing the test build.

If you decide it's ok, can you flag for sr=? from henry.jia for me?

Comment 13

[Henry Jia]

Comment on attachment 151663 [details] [diff] [review]
Better patch

sr=Henry

Comment 14

[Aaron Leventhal]

Checking in src/atk/nsXULTreeAccessibleWrap.h;
/cvsroot/mozilla/accessible/src/atk/nsXULTreeAccessibleWrap.h,v  <-- 
nsXULTreeAccessibleWrap.h
new revision: 1.3; previous revision: 1.2
done
Checking in src/atk/Makefile.in;
/cvsroot/mozilla/accessible/src/atk/Makefile.in,v  <--  Makefile.in
new revision: 1.8; previous revision: 1.7
done
Checking in src/xul/Makefile.in;
/cvsroot/mozilla/accessible/src/xul/Makefile.in,v  <--  Makefile.in
new revision: 1.14; previous revision: 1.13
done
Checking in src/xul/nsXULTreeAccessible.cpp;
/cvsroot/mozilla/accessible/src/xul/nsXULTreeAccessible.cpp,v  <-- 
nsXULTreeAccessible.cpp
new revision: 1.21; previous revision: 1.20
done
Checking in src/xul/nsXULTreeAccessible.h;
/cvsroot/mozilla/accessible/src/xul/nsXULTreeAccessible.h,v  <-- 
nsXULTreeAccessible.h
new revision: 1.12; previous revision: 1.11
done
Checking in src/msaa/Makefile.in;
/cvsroot/mozilla/accessible/src/msaa/Makefile.in,v  <--  Makefile.in
new revision: 1.6; previous revision: 1.5
done
RCS file: /cvsroot/mozilla/accessible/src/msaa/nsXULTreeAccessibleWrap.cpp,v
done
Checking in src/msaa/nsXULTreeAccessibleWrap.cpp;
/cvsroot/mozilla/accessible/src/msaa/nsXULTreeAccessibleWrap.cpp,v  <-- 
nsXULTreeAccessibleWrap.cpp
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/accessible/src/msaa/nsXULTreeAccessibleWrap.h,v
done
Checking in src/msaa/nsXULTreeAccessibleWrap.h;
/cvsroot/mozilla/accessible/src/msaa/nsXULTreeAccessibleWrap.h,v  <-- 
nsXULTreeAccessibleWrap.h
initial revision: 1.1
done
Checking in src/mac/nsAccessibleWrap.h;
/cvsroot/mozilla/accessible/src/mac/nsAccessibleWrap.h,v  <--  nsAccessibleWrap.h
new revision: 1.5; previous revision: 1.4
done
Checking in src/other/nsAccessibleWrap.h;
/cvsroot/mozilla/accessible/src/other/nsAccessibleWrap.h,v  <--  nsAccessibleWrap.h
new revision: 1.4; previous revision: 1.3
done

Comment 15

[Firefox Bug Husbandry Bot]

https://bugzilla.mozilla.org/show_bug.cgi?id=1527381