Last Comment Bug 658272 - Implement figure and figcaption accessibility
: Implement figure and figcaption accessibility
Status: VERIFIED FIXED
:
Product: Core
Classification: Components
Component: Disability Access APIs (show other bugs)
: unspecified
: All All
: -- normal (vote)
: mozilla11
Assigned To: alexander :surkov
:
Mentors:
Depends on: 573356 765172
Blocks: html5a11y
  Show dependency treegraph
 
Reported: 2011-05-19 08:09 PDT by David Bolter [:davidb] ***PTO until 29th***
Modified: 2012-06-19 11:03 PDT (History)
8 users (show)
mzehe: in‑testsuite+
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
patch (15.57 KB, patch)
2011-11-22 01:59 PST, alexander :surkov
mzehe: review+
Details | Diff | Splinter Review
patch2 (21.04 KB, patch)
2011-11-29 20:44 PST, alexander :surkov
tbsaunde+mozbugs: review+
Details | Diff | Splinter Review

Description David Bolter [:davidb] ***PTO until 29th*** 2011-05-19 08:09:47 PDT
This is an unusual case really. The figure element can contain a figcaption element that describes its siblings (also children of the figure element).

Related W3 bug: http://www.w3.org/Bugs/Public/show_bug.cgi?id=12702
Comment 1 alexander :surkov 2011-11-04 12:45:47 PDT
TPG: http://www.paciellogroup.com/blog/2011/08/html5-accessibility-chops-the-figure-and-figcaption-elements/
w3c: http://dev.w3.org/html5/spec/grouping-content.html#the-figure-element
example: http://www.w3schools.com/html5/tryit.asp?filename=tryhtml5_figure

I think we should request new IA2 role for figure and expose the accessible labelled_by figcaption having caption role. So AT seeing the figure accessible can announce this is a figure and figure name and then the user can go into the figure content or skip it. Sounds good?
Comment 2 David Bolter [:davidb] ***PTO until 29th*** 2011-11-04 12:51:04 PDT
(In reply to alexander surkov from comment #1)
> Sounds good?

Yes.
Comment 3 David Bolter [:davidb] ***PTO until 29th*** 2011-11-04 12:53:32 PDT
Steve what do you think of comment 1?
Comment 4 steve faulkner 2011-11-04 14:06:40 PDT
(In reply to David Bolter [:davidb] from comment #3)
> Steve what do you think of comment 1?

Hi Dave, I am in complete agreement with alex.
This is the conclusion i reached from writing and getting feedback on http://www.paciellogroup.com/blog/2011/08/html5-accessibility-chops-the-figure-and-figcaption-elements/

We have discussed and appeared to be in agreement in the PF F2F this week about adding a figure role to ARIA 1.1 So in short, what is proposed is a good approach.
Comment 5 alexander :surkov 2011-11-09 01:41:13 PST
Good to know, Steve. I filed request to IA2 group - http://lists.linuxfoundation.org/pipermail/accessibility-ia2/2011-November/001550.html
Comment 6 alexander :surkov 2011-11-09 02:07:58 PST
on ATK mailing list: http://mail.gnome.org/archives/gnome-accessibility-devel/2011-November/msg00006.html
Comment 7 Aaron Leventhal 2011-11-15 14:47:21 PST
Looks exactly like fieldset/legend to me, but with a different name. I'd reuse the same implementation and expose it as ROLE_GROUP with an accessible name. We also have the ROLE_CAPTION and label relations. IIRC we used ROLE_GROUP for more than just fieldset anyway -- it's any kind of group where we needed that.

It's worth trying and seeing how screen readers already work with it. From a user's point of view this is just another group of content. If they work well already, then a lot of pain is saved in terms of "Can I Use" backward compat concerns etc.

If AT wants to differentiate further from those those two kinds of groups, in order to announce a friendlier role, like "Figure", then they have the object attribute "tag" available. Newer ATs could use that, but older ATs would still work.

I'm trying to figure out why we'd want to do more work than that.
Comment 8 alexander :surkov 2011-11-15 20:51:27 PST
I wouldn't agree the figure is just another group, it provides richer semantics. From user point of view the figure is a group of elements used to be referred from text content and illustrate it (what can be treated wide). I think the figure semantics should be exposed to AT.

So we have two alternatives:
1) Use ROLE_GROUP and expose tag object attribute (or xml-roles if suites) as you suggested. I agree this is great for backward compatibility. Older AT can announce this is a group and its name but no information that this is a figure what is incomplete information.
2) Introduce new role as I suggested. Older ATs miss this information. On the other hand HTML5 is still new standard and currently actively developing by both browsers and ATs what means we can't have backward compatibility for every piece of HTML5. So this is a right time to introduce new roles, new states and everything we need for better accessibility. Maybe not for this case but anyway I didn't hear back from AT developers yet on this issue.

Actually this point is raised not for the first time. I think we should suggest a new method for AT APIs to expose hierarchy of roles like figure -> group -> section so that AT can pick up the role they knows best. This is out of context of this bug though.
Comment 9 Aaron Leventhal 2011-11-16 14:37:57 PST
Based on my experience getting things adopted by AT vendors and web developers, I recommend route #1. Otherwise, you end up with a chicken and egg problem where no one uses the new markup because few ATs support it, and the ATs don't support it because no one uses the markup, etc. Finally, #1 has the advantage that users don't have to upgrade their expensive software to get some benefits.

I'd vote for adoption and immediacy over the pure approach. If we took the pure approach just for purity's sake we would not have gotten as far as we did with AT and author support of ARIA, let alone HTML :)

OS X even has a similar concept -- subroles. The role is exposed for the older version of the AT, and the subrole is more specific.

My 2 cents.
Comment 10 James Teh [:Jamie] 2011-11-16 14:59:47 PST
Two things:
1. The compatibility issue depends on how well ATs handle unknown roles. I would think that most wouldn't just ignore unknown roles altogether and would still try to render the content. This would need to be tested, though.
2. Most ATs don't say anything specific for groupings (fieldsets) when browsing a document. If I'm right, this means there'll be no difference between an unknown role (assuming 1) is correct) and a grouping. Again, this would need to be tested.

Imo, relying on the "tag" object attribute is generally a bad idea because it is non-standard and makes the functionality HTML specific. If we're going to rely on any object attribute, it should be xml-roles and the role name should be made part of some spec.
Comment 11 steve faulkner 2011-11-16 15:08:43 PST
Hi Aaron, I don't think the markup's use by web developers is predicated on whether its accessibility supported and I think figure/figcaption semantics (as used and as specced) are and should be differentiated from fieldset/legend.

I don't see any major disagreement here though. Implement now as role group (as there is no better mapping) its more generic but at least its something, pursue getting the figure role defined in IA2 and ARIA 1.1. when and if defined it can be used.
Comment 12 steve faulkner 2011-11-16 15:12:57 PST
(In reply to James Teh [:Jamie] from comment #10)
> Two things:
> 1. The compatibility issue depends on how well ATs handle unknown roles. I
> would think that most wouldn't just ignore unknown roles altogether and
> would still try to render the content. This would need to be tested, though.
> 2. Most ATs don't say anything specific for groupings (fieldsets) when
> browsing a document. If I'm right, this means there'll be no difference
> between an unknown role (assuming 1) is correct) and a grouping. Again, this
> would need to be tested.
> 

both JAWS and window eyes have distinct behaviours for fieldset and legend.
This article i wrote is oldbut provides and overview http://www.paciellogroup.com/blog/2007/11/fieldsets-legends-and-screen-readers/
Comment 13 Aaron Leventhal 2011-11-16 16:26:50 PST
First, no major disagreement. Not like WHATWG/HTML5-ish anyway. If I sound ornery it's because I'm in a hurry or tired :) You guys are my peoples.

ROLE_GROUP is supported. I remember working with a very large AT company on support for ROLE_GROUP. I know it's used for forms mode. As you enter a group the name of the group is read with whatever you land on. That's how I remember it. I'm not sure about browse/virtual mode.

Our approach in the past was to maximize compatibility, because compatibility issues are absolute poison to adoption of markup, in general. Look at longdesc, or for that matter, XHTML 1.0. Who wants to read future blog posts about which screen readers work alright with figcaption?

We needed to get 4 screen readers to work with us: JAWS, WE, NVDA and Orca. And, we needed them to work as soon as possible, as well as possible. 

We could have tried to add new roles to IA2 and ATK all the time. Getting those aligned is not easy in the first place.

To support the web's ever growing list of surprises, any IDL-based API is going to need some loose attributes. We made the decision to be ok with that, and it paid off a lot. 

Is it that bad? We documented our object attribute and role usage so that others could follow it. We knew WebKit would come along and need it at some point. We also build Speclenium so that they could look for differences in their implementation.

It's hard to explain all the complex dominoes that fall one way or another when there is more vs. less compatibility for a given piece of markup, with current ATs. But, my advice is still to just be consistent. You can document the object attribute and role usage, and be more compatible for more users, who love it when stuff just works.

Object attribute usage in Gecko:
https://developer.mozilla.org/en/Accessibility:AT-APIs:Gecko:Attrs
HTML to role/state mappings in GEcko:
https://developer.mozilla.org/en/Accessibility/AT-APIs/AT-SPI_HTML_Examples
Comment 14 James Teh [:Jamie] 2011-11-16 16:35:40 PST
First, in general, I agree with using existing roles where they fit. The question is really whether grouping fits for figures.

Grouping certainly fits for fieldset/legend because it is a "grouping" of related interactive controls. That is precisely how a grouping works in a standard desktop application. Thus, nothing is being fudged; it just "fits". There's also nothing special about the grouping as far as the user is concerned; it is just a group.

Whether it fits for figure is debatable. A figure isn't a "grouping of related interactive controls". If you take out interactive controls, it probably fits. However, there are special semantics that do matter to the user. The user probably does care that it is a figure. It is not an arbitrary grouping. Personally, I'd argue figure is closer to table than grouping in terms of semantics.

All of this said, I'm not saying usage of object attributes is a bad thing. I'm saying usage of the tag object attribute is a bad thing.
Comment 15 Aaron Leventhal 2011-11-16 19:40:38 PST
Ok. Right, it's debatable. I kind of see roles as inheriting from each other, like classes. the getRole method is really just returning the base class. Other stuff describes it further. There are object attributes, or we can use these
BSTR 	extendedRole [get]
 	Returns the extended role.
BSTR 	localizedExtendedRole [get]
 	Returns the localized extended role. 
But that's just in IA2. I don't remember what ATK/AT-SPI have or how we use those in IA2, if at all. Worth investigating.

Something I always have on my mind is the pain we had whenever we had to ask the commercial vendors to do something. Any time we could, we made sure we didn't rely on it. There was a very high cost, and not just in emotional pain. You might be shocked to know. And, Mozilla did not bear the brunt of this. Because it's outside of our control, sometimes we have to be clever. It's not the clean way, or the standard way, or the right way, but, it's a huge reason for the pragmatic approach.

This leads to a question -- is the expected AT behavior different for navigating to a link inside a figure vs. a control inside a fieldset. Wouldn't you want to announce the name of the fieldset/figure as you entered?

IOW is the difference more conceptual or more practical?
Comment 16 alexander :surkov 2011-11-16 19:43:41 PST
So
1) go with group role as Steve suggested this is a best what we can do today
2) make IA2 and ATK introduce new role for figure since group semantics isn't well suitable for figure
3) request role hierarchy API for IA2 and ATK so we never fall into chicken-egg problem in the future
3.1) OS X approach subrole is backward compatible. So new AT relies on subrole, role is a fallback.
3.2) reverse approach to subrole. I like it more since it sounds cleaner from API point of view and doesn't require existing AT to do huge code changes. Not backward compatible.
3.3) same as 3.2, provides a list of roles in hierarchy.

If we do 1-2-3.1 then we are backward compatible and users are happy
If we do 1-2-3.2(3.3) then at some point we break backward compatibility. That's not a problem for NVDA, Orca users but sort of problem for commercial screen reader users. But maybe it's not big deal for those users since long time may be required before this markup is picked up by wild web.

Thoughts?
Comment 17 alexander :surkov 2011-11-16 19:49:36 PST
(In reply to Aaron Leventhal from comment #15)

> This leads to a question -- is the expected AT behavior different for
> navigating to a link inside a figure vs. a control inside a fieldset.
> Wouldn't you want to announce the name of the fieldset/figure as you entered?
> 
> IOW is the difference more conceptual or more practical?

Jamie said that figure is more closer to table semantics rather than group semantics. From practical point of view I think the user hears that this is a figure and its name and he may decide to not go into figure, just skip it. I think that's a difference from group element where user gets into the group and hears he's inside of group.
Comment 18 James Teh [:Jamie] 2011-11-16 19:56:55 PST
In Nvda at least, you can't actually skip past an element at present.

I guess the real question is whether there are likely to be interactive elements in figures. If yes, that would probably make them more like fieldsets where the caption needs to be read when they are focused.

Aaron, I take your point re backwards compatibility. However, as I said above in comment #10, it may not break; the only way to know is to test with current ATs.

If the consensus is that figures have the same semantics as fieldsets, then by all means just expose it in xml-roles and use the grouping role. However, if this is just "fudging", well... older ATs won't have support for figures anyway, so the question is whether older ATs treating them as fieldsets is actually correct.
Comment 19 alexander :surkov 2011-11-16 19:57:41 PST
(In reply to Aaron Leventhal from comment #15)

> BSTR 	extendedRole [get]
>  	Returns the extended role.
> BSTR 	localizedExtendedRole [get]
>  	Returns the localized extended role. 
> But that's just in IA2. I don't remember what ATK/AT-SPI have or how we use
> those in IA2, if at all. Worth investigating.

right, that's OS X approach from 3.1 (comment #16)
Comment 20 Aaron Leventhal 2011-11-16 21:24:52 PST
At a minimum figures will have links -- to sources, more info, etc.

In addition, a figure can have a diagram or chart, two things that developers like to make interactive. It's common to allow items in a diagram to be clicked for more info, e.g. stats. Many diagrams and charts have interactive elements for controlling the view. For example, the Flash diagram at http://gs.statcounter.com/ has buttons to toggle which lines are shown, and radio button options for line vs. bar chart. There are also combo boxes for controlling region and other options.
Comment 21 alexander :surkov 2011-11-16 21:28:42 PST
I think the point is not that figure can contain interactive elements but the point is figure is sort of illustration, addition or clarification to main text so it's optional and can be skipped by user.
Comment 22 Aaron Leventhal 2011-11-16 21:50:03 PST
Surkov, regarding "optional and can be skipped" -- isn't all content optional and can be skipped? :) I might want to skip past any kind of group of related information, if I'm not interested in what's in that group.
Comment 23 alexander :surkov 2011-11-16 22:07:55 PST
(In reply to Aaron Leventhal from comment #22)
> Surkov, regarding "optional and can be skipped" -- isn't all content
> optional and can be skipped? :) I might want to skip past any kind of group
> of related information, if I'm not interested in what's in that group.

yeah, right but the way you skip may be different. Please correct me I'm wrong but if you arrow down the content and you see a table then next arrow down doesn't go into table (to go into it you should press something like ctrl+arrow down), in case of group you arrow down into control of the group and get announced you're inside of group. Sure that depends on AT but anyway elements may have different level of skipability :) I think this is one reason why figure and fieldset may be different from your point of view (except 'group' and 'figure' announcements) and why ROLE_GROUP doesn't really suites it well.
Comment 24 Kevin Chao 2011-11-16 22:12:42 PST
While it’s very true that figure and figcaption are not very common on the interweb. With the work I do, both professionally and academically, I work with a lot of eTextbooks that contain lots of figure and figcaption. I work with both the eTextbook reader and content accessibility taggers/remediators. 

Currently, the graphic, which is the figure will contain the alttext, below that will be the caption, and that’s determined by the textbook author. We have control over the altext for figure, not the figcaption.

Due to figure and figcaption still being worked out for ARIA 1.1, we are currently doing the following: 
Graphic alttext
Caption: author text
Note: 
Sometimes figure numbers will be above graphic, e.g. figure 1.2;
With figure being a graphic , it’s possible to use NVDA and press g or SHIFT+G to go to next/previous graphic;
Alttext we put in will try not to repeat figcaption, but augment/supplement the figure by adding useful information that’s not contained in text of figcaption;
Figcaption is noted as “Caption:”;
Figcaption will contain figure number and author text.

Once ARIA 1.1 with figure and figcaption is all sorted and finalized, we’ll work on getting it built into eTextbook reader and content accessibility tagging/remediation process.
Comment 25 James Teh [:Jamie] 2011-11-16 22:16:33 PST
In short, I don't really care strongly enough either way. I'm happy with either of these two solutions:
1. Create figure role; or
2. Expose figure role in xml-roles object attribute and use role of grouping.
Comment 26 alexander :surkov 2011-11-22 01:59:43 PST
Created attachment 576107 [details] [diff] [review]
patch

expose figure as grouping and xml-roles:figure
Comment 27 steve faulkner 2011-11-22 02:16:36 PST
(In reply to alexander surkov from comment #26)
> Created attachment 576107 [details] [diff] [review] [diff] [details] [review]
> patch
> 
> expose figure as grouping and xml-roles:figure

whats the status of the mapping of figcaption content as the accessible name for figure?
Comment 28 alexander :surkov 2011-11-22 02:19:28 PST
(In reply to steve faulkner from comment #27)
> (In reply to alexander surkov from comment #26)
> > Created attachment 576107 [details] [diff] [review] [diff] [details] [review] [diff] [details] [review]
> > patch
> > 
> > expose figure as grouping and xml-roles:figure
> 
> whats the status of the mapping of figcaption content as the accessible name
> for figure?

sure, it works together with labelled_by/label_for relations. by that message I outlined the chosen approach.
Comment 29 Marco Zehe (:MarcoZ) 2011-11-22 06:55:46 PST
Comment on attachment 576107 [details] [diff] [review]
patch

r=me for the tests and the functionality. With NVDA, this works fine. And it would give NVDA and Orca the chance to also announce something special if they so desire.
Comment 30 Trevor Saunders (:tbsaunde) 2011-11-29 19:38:48 PST
Comment on attachment 576107 [details] [diff] [review]
patch


>+  if (tag == nsGkAtoms::figcaption) {
>+    nsAccessible* accessible =
>+      new nsHTMLFigcaptionAccessible(aContent, aWeakShell);
>+    NS_IF_ADDREF(accessible);
>+    return accessible;
>+  }
>+
>+  if (tag == nsGkAtoms::figure) {
>+    nsAccessible* accessible = new nsHTMLFigureAccessible(aContent, aWeakShell);
>+    NS_IF_ADDREF(accessible);
>+    return accessible;
>+  

I think this whole bunch of ifs would be nicer as a switch something like

switch(tag) {
case nsGkAtoms::figure:
  accessible = new FigureAccessible();
  break;
case nsGkAtoms::figcaption:
  accessible = new FigCaptionAccessible();
  break;
default:
  break;
}

but I'd be fine with cleaning this up later.

>+nsHTMLFigureAccessible::NativeRole()
>+{
>+  return nsIAccessibleRole::ROLE_GROUPING;

I think I'd prefer we use a figure role internally and map it to grouping when we map to platform roles, but I don't really care that much.

>+  nsIContent* childContent = nsnull;
>+  while ((childContent = mContent->GetChildAt(count++))) {

I thought the DOM people prefered we use prev / next sibling instead of ChildAt()?

>+    if (childContent->NodeInfo()->Equals(nsGkAtoms::figcaption,
>+                                         mContent->GetNameSpaceID())) {

why isn't this just nsIContent::Tag() == figcaption?

>+  if (figure &&
>+      figure->GetContent()->NodeInfo()->Equals(nsGkAtoms::figure,
>+                                               mContent->GetNameSpaceID())) {

same
Comment 31 alexander :surkov 2011-11-29 19:59:51 PST
(In reply to Trevor Saunders (:tbsaunde) from comment #30)

> switch(tag) {
> case nsGkAtoms::figure:
>   accessible = new FigureAccessible();
>   break;
> case nsGkAtoms::figcaption:
>   accessible = new FigCaptionAccessible();
>   break;
> default:
>   break;
> }

I never used switch on pointers. Is just syntax issue?

> but I'd be fine with cleaning this up later.

ok

> >+nsHTMLFigureAccessible::NativeRole()
> >+{
> >+  return nsIAccessibleRole::ROLE_GROUPING;
> 
> I think I'd prefer we use a figure role internally and map it to grouping
> when we map to platform roles, but I don't really care that much.

rigth, I originally intended do that but eventually I missed it.

> >+  nsIContent* childContent = nsnull;
> >+  while ((childContent = mContent->GetChildAt(count++))) {
> 
> I thought the DOM people prefered we use prev / next sibling instead of
> ChildAt()?

good catch, copy/paste issue

> >+    if (childContent->NodeInfo()->Equals(nsGkAtoms::figcaption,
> >+                                         mContent->GetNameSpaceID())) {
> 
> why isn't this just nsIContent::Tag() == figcaption?

just checking namespace, making sure, it's not comping from different markup
Comment 32 alexander :surkov 2011-11-29 20:44:15 PST
Created attachment 577845 [details] [diff] [review]
patch2
Comment 33 alexander :surkov 2011-12-05 01:45:54 PST
inbound land http://hg.mozilla.org/integration/mozilla-inbound/rev/e89b8ace0460
Comment 34 Matt Brubeck (:mbrubeck) 2011-12-05 10:33:52 PST
https://hg.mozilla.org/mozilla-central/rev/e89b8ace0460
Comment 35 Marco Zehe (:MarcoZ) 2011-12-06 08:45:23 PST
Verified fixed in Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0a1) Gecko/20111206 Firefox/11.0a1

Note You need to log in before you can comment on or make changes to this bug.