249998 - Make DHTML accessible

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect, P1)

Description

[Aaron Leventhal]

Full explanation at
http://www.mozilla.org/projects/ui/accessibility/dynamic-accessibility.html

Dynamic web content is not accessible, because it uses vanilla <div>'s and
<span>'s combined with Javascript rather than well known declarative markup to
describe the behavior of individual widgets, such as menus and tree views.
Assistive technologies have no insight as to what this Javascript is doing, what
these widgets are supposed to be, or what state they're in or what they're
capable of. These widgets are usually not even keyboard focusable, partially
because authors don't concern themselves with that problem, but also because
there is no programmatic way to set focus to just any element in a page.

We need to allow dhtml authors to add role and state semantics to their custom
elements, and use that information to expose that information via MSAA and ATK.

Comment 1

[Aaron Leventhal]

Here's the rough design:

(The following happens only when accessibility is active):

1. When new document loaded, check for presense of dhtml accessibility namespace
(we need to create one)
   If present, get namespace ID constant for it.

2. While walking the DOM tree and creating accessibles for some nodes, check for
the role attribute with cached namespace ID.
   If present, create an nsGenericAccessible for it.

3. Make nsGenericAccessible::GetRole() convert the role attribute in the dhtml
accessibility namespace to
   the appropriate constant from nsIAccessible.
   Implementation: should we look up the string to constant role conversion in a
hash table keyed on the string name?

4. Make nsGenericAccessible::GetState() convert the various new state attributes
in the dhtml accessibility namespace.
   Implementation options (which is faster)?
   1) iterate through all of the attributes on the current element. If it has
the correct namespace,
   look the attribute name up in a hash table that maps to the correct
nsIAccessible state attribute
   2) Go through the list of new state attributes, and call GetAttributeNS for each.

5. Implement nsGenericAccessible::TakeFocus() to set focus on the elemen when it
has STATE_FOCUSABLE.

6. Implement nsGenericAccessible::GetValue() to get JS value property from any
element. How can we do that? Can we fire an event whenever the value changes?

Comment 2

[Aaron Leventhal]

Ideally the role and state conversions would be loaded from xml or rdf.
RDF doesn't sound like a real option sice it's not supported on minimo. Suggestions?

Comment 3

[Aaron Leventhal]

Attachment: Checkbox testcase

Perhaps someone can help me figure out what's wrong with the namespace markup.
The checkbox should be starting out checked and it isn't.

Comment 4

[Aaron Leventhal]

Better to use the URL
(http://www.moonset.net/aaronwork/tests/dhtml/checkbox-with-ns.html) to debug
the test case since the images are included there.

Comment 5

[Aaron Leventhal]

Attachment: Rough plan -- seeking feedback

Comment 6

[Johnny Stenback (:jst)]

Comment on attachment 153035 [details]
Rough plan -- seeking feedback

> Make nsGenericAccessible::GetState() convert the various new state attributes
>in the dhtml accessibility namespace.
>   *** Implementation options (which is faster)?
>   a) iterate through all of the attributes on the current element.
> If it has the correct namespace,
>   look the attribute name up in a hash table that maps to the
> correct nsIAccessible state attribute
>   b) Go through the list of new state attributes, and call
>   GetAttributeNS for each.

That depens on what number is greater, the number of attributes on an average
HTML element, or the number of new state attributes. If the number of new state
attributes is greater (which is what I would guess), then I'd recommend walking
through the attributes on the element (using nsIContent::GetAttrNameAt()), and
then calling nsIContent::GetAttr() if there's a match.

> Implement nsGenericAccessible::GetValue() to get JS value property
> from any element. *** How can we do that? *** Also, can we fire an
> event whenever the value changes?

Getting the value is easy, but you'll add a dependency on the JS engine. You'll
need to wrap the DOM object with XPConnect, get the JS object and call
JS_GetProperty() on it. There's no easy way to get notifications about changes
to JS properties.

>*** Ideally the role and state conversions would be loaded from a data source rather than hard coded. We could load the data source lazily.
>RDF doesn't sound like a real option sice it's not supported on minimo, and apparanently Mozilla's RDF impl is very different from W3C's RDF. Suggestions?

Raw XML file?

Comment 7

[Aaron Leventhal]

Attachment: Not finished yet. Create accessible object depending on role attribute

Exposes <span xa:role="checkbox">My checkbox</span> as a checkbox. Window-Eyes
says "checkbox not checked" as you tab to it.
Example here:
http://www.moonset.net/aaronwork/tests/dhtml/checkbox-with-ns.xhtml

Comment 8

[Aaron Leventhal]

Attachment: Support roles, states and accessibility events for changed states

Note that this code will still undergo significant changes. This is the initial
push.

Also, I don't like having both GetRole/GetFinalRole, GetState/GetFinalState and
GetValue/GetFinalValue in nsIAccessible, but that was the best way to avoid a
HUGE patch. We can narrow things down to a single method for each of
role/state/value in a different bug.

Here are the specs for what we're doing:
http://www.w3.org/WAI/PF/Group/roadmap/
and http://www.w3.org/WAI/PF/Group/html-state/

Note that using a role attribute only affects the role which gets exposed via
GetRole, not the interfaces that the object supports.

Comment 9

[Philip K. Warren]

Comment on attachment 172298 [details] [diff] [review]
Support roles, states and accessibility events for changed states

>Index: accessible/public/nsIAccessible.idl

Please change the UUID in this file after making the changes.

>+   * This does not take into acount xhtml2:role as the finalRole does.

'account'

>Index: accessible/public/nsPIAccessibleDocument.idl

Please change the UUID.

>Index: accessible/src/base/nsAccessible.cpp
>===================================================================
>RCS file: /cvsroot/mozilla/accessible/src/base/nsAccessible.cpp,v
>retrieving revision 1.121
>diff -p -u -5 -r1.121 nsAccessible.cpp
>--- accessible/src/base/nsAccessible.cpp	19 Jan 2005 07:10:37 -0000	1.121
>+++ accessible/src/base/nsAccessible.cpp	24 Jan 2005 23:33:44 -0000
>@@ -220,10 +221,43 @@ NS_IMETHODIMP nsAccessible::SetNextSibli
> {
>   mNextSibling = aNextSibling? aNextSibling: DEAD_END_ACCESSIBLE;
>   return NS_OK;
> }
> 
>+NS_IMETHODIMP nsAccessible::Init()
>+{
>+  nsCOMPtr<nsIContent> content(do_QueryInterface(mDOMNode));
>+  nsAutoString roleString;
>+  if (content &&
>+      NS_CONTENT_ATTR_HAS_VALUE == content->GetAttr(kNameSpaceID_XHTML2_Unofficial, 
>+                                                    nsAccessibilityAtoms::role, 
>+                                                    roleString)) {
>+    // QI to nsIDOM3Node causes some overhead. Unfortunately we need to do this each
>+    // time there is a role attribute, because the prefixe to namespace mappings
>+    // can change within any subtree via the xmlns attribute
>+    nsCOMPtr<nsIDOM3Node> dom3Node(do_QueryInterface(content));

Does this QI always succeed? If not you will crash below.

>@@ -1253,10 +1287,120 @@ NS_IMETHODIMP nsAccessible::FireToolkitE
>+nsRoleMapEntry nsAccessible::gWAIRoleMap[] = 
>+{
>+  // Eventually we will most likely be loading an RDF file that contains this information
>+  // Using RDF will also allow for role extensibility
>+  // XXX Should we store attribute names in this table as atoms instead of strings?
>+  {"button", ROLE_PUSHBUTTON, 0, {0, 0, 0}, {0, 0, 0}, {0, 0, 0}},
>+  {"checkbox", ROLE_CHECKBUTTON, 0, {"checked", "true", STATE_CHECKED}, {"readonly", 0, STATE_READONLY}, {0, 0, 0}},

Do you explain the meaning of these entries in the map anywhere? Please open a
bug for getting these from an external source (e.g. RDF).

>+nsStateMapEntry nsAccessible::gDisabledStateMap = {"disabled", 0, STATE_UNAVAILABLE };
>+// Possibly split into 2 kinds of roles -- those that hold data and those that don't
>+// These states only apply to items that can hold data
>+//nsStateMapEntry nsAccessible::gInvalidStateMap = {"invalid", 0, STATE_INVALID }; // XXX wait until extended states fix lands
>+//nsStateMapEntry nsAccessible::gRequiredStateMap = {"required", 0, STATE_REQUIRED }; // XXX no MSAA or ATK mapping

What do these nsStateMapEntry structs contain?

>Index: accessible/src/base/nsAccessible.h
>===================================================================
>RCS file: /cvsroot/mozilla/accessible/src/base/nsAccessible.h,v
>retrieving revision 1.51
>diff -p -u -5 -r1.51 nsAccessible.h
>--- accessible/src/base/nsAccessible.h	18 Aug 2004 14:39:44 -0000	1.51
>+++ accessible/src/base/nsAccessible.h	24 Jan 2005 23:33:44 -0000
>@@ -54,10 +54,27 @@ class nsIDOMNode;
>+struct nsRoleMapEntry
>+{
>+  const char *roleString; // such as "button"
>+  PRUint32 role;   // use this role
>+  PRUint32 state;  // always OR state with this
>+  nsStateMapEntry attributeMap1;
>+  nsStateMapEntry attributeMap2;
>+  nsStateMapEntry attributeMap3;
>+};
>+

What do attributeMap[1-3] contain?

My only major concerns with this patch are that I don't really like the *Final*
naming scheme. I think that we should come up with a clearer way of designating
that GetFinalState is the accessible state plus any additional state
information from the XHTML2 state attribute.

Comment 10

[Aaron Leventhal]

Attachment: Addresses pkw's comments, except for names GetFinalFoo(), because those are temporary

I will file bugs on using RDF and removing GetFinalFoo() methods.

Comment 11

[Aaron Leventhal]

Comment on attachment 172470 [details] [diff] [review]
Addresses pkw's comments, except for names GetFinalFoo(), because those are temporary

Even though this has r=pkw, could someone from Sun please still take a look?
It's an important change that should be understood by the Sun team.

Comment 12

[Johnny Stenback (:jst)]

Comment on attachment 172470 [details] [diff] [review]
Addresses pkw's comments, except for names GetFinalFoo(), because those are temporary

- In nsDocAccessible:

+    if (!mEventsToFire) {
+      NS_NewArray(getter_AddRefs(mEventsToFire));
+      NS_ENSURE_TRUE(mEventsToFire, NS_ERROR_OUT_OF_MEMORY);

mEventsToFire should be an nsCOMArray... see below...

[...]
+    mEventsToFire->QueryElementAt(index, NS_GET_IID(nsIAccessibleEvent),
+      getter_AddRefs(accessibleEvent));

... then that could change to:

     accessibleEvent = mEventsToFire[index];

[...]
+    nsCOMPtr<nsIMutableArray> mEventsToFire;

Make that nsCOMArray<nsIAccessibleEvent> and you'll save yourself some typing
and memory heap overhead.

sr=jst

Comment 13

[Aaron Leventhal]

/cvsroot/mozilla/accessible/public/nsIAccessible.idl,v  <--  nsIAccessible.idl
new revision: 1.31; previous revision: 1.30
done
Checking in accessible/public/nsPIAccessibleDocument.idl;
/cvsroot/mozilla/accessible/public/nsPIAccessibleDocument.idl,v  <-- 
nsPIAccessibleDocument.idl
new revision: 1.2; previous revision: 1.1
done
Checking in accessible/src/atk/nsAccessibleWrap.cpp;
/cvsroot/mozilla/accessible/src/atk/nsAccessibleWrap.cpp,v  <-- 
nsAccessibleWrap.cpp
new revision: 1.21; previous revision: 1.20
done
Checking in accessible/src/base/nsAccessibilityAtomList.h;
/cvsroot/mozilla/accessible/src/base/nsAccessibilityAtomList.h,v  <-- 
nsAccessibilityAtomList.h
new revision: 1.9; previous revision: 1.8
done
Checking in accessible/src/base/nsAccessible.cpp;
/cvsroot/mozilla/accessible/src/base/nsAccessible.cpp,v  <--  nsAccessible.cpp
new revision: 1.122; previous revision: 1.121
done
Checking in accessible/src/base/nsAccessible.h;
/cvsroot/mozilla/accessible/src/base/nsAccessible.h,v  <--  nsAccessible.h
new revision: 1.52; previous revision: 1.51
done
Checking in accessible/src/base/nsBaseWidgetAccessible.cpp;
/cvsroot/mozilla/accessible/src/base/nsBaseWidgetAccessible.cpp,v  <-- 
nsBaseWidgetAccessible.cpp
new revision: 1.31; previous revision: 1.30
done
Checking in accessible/src/base/nsDocAccessible.cpp;
/cvsroot/mozilla/accessible/src/base/nsDocAccessible.cpp,v  <--  nsDocAccessible.cpp
new revision: 1.49; previous revision: 1.48
done
Checking in accessible/src/base/nsDocAccessible.h;
/cvsroot/mozilla/accessible/src/base/nsDocAccessible.h,v  <--  nsDocAccessible.h
new revision: 1.18; previous revision: 1.17
done
Checking in accessible/src/html/nsHTMLSelectAccessible.cpp;
/cvsroot/mozilla/accessible/src/html/nsHTMLSelectAccessible.cpp,v  <-- 
nsHTMLSelectAccessible.cpp
new revision: 1.43; previous revision: 1.42
done
Checking in accessible/src/msaa/nsAccessibleWrap.cpp;
/cvsroot/mozilla/accessible/src/msaa/nsAccessibleWrap.cpp,v  <-- 
nsAccessibleWrap.cpp
new revision: 1.15; previous revision: 1.14
done
Checking in accessible/src/xul/nsXULMenuAccessible.cpp;
/cvsroot/mozilla/accessible/src/xul/nsXULMenuAccessible.cpp,v  <-- 
nsXULMenuAccessible.cpp
new revision: 1.29; previous revision: 1.28
done
Checking in content/base/public/nsINameSpaceManager.h;
/cvsroot/mozilla/content/base/public/nsINameSpaceManager.h,v  <-- 
nsINameSpaceManager.h
new revision: 3.27; previous revision: 3.26
done
Checking in content/base/src/nsNameSpaceManager.cpp;
/cvsroot/mozilla/content/base/src/nsNameSpaceManager.cpp,v  <-- 
nsNameSpaceManager.cpp
new revision: 3.53; previous revision: 3.52
done

Comment 14

[Firefox Bug Husbandry Bot]

https://bugzilla.mozilla.org/show_bug.cgi?id=1527381