[Accessibility] Chat partner status changes (typing, away, etc.) are not accessible

RESOLVED WORKSFORME

Status

Instantbird
Conversation
RESOLVED WORKSFORME
4 years ago
4 years ago

People

(Reporter: Instantbot, Unassigned)

Tracking

Details

Attachments

(3 attachments)

(Reporter)

Description

4 years ago
*** Original post on bio 409 by Joanmarie <joanmarie.diggs AT gmail.com> at 2010-04-30 22:58:00 UTC ***

*** Due to BzAPI limitations, the initial description is in comment 1 ***
(Reporter)

Comment 1

4 years ago
Created attachment 8352043 [details]
Screenshot 1: Accerciser's event monitor

*** Original post on bio 409 as attmnt 302 by joanmarie.diggs AT gmail.com at 2010-04-30 22:58:00 UTC ***

Steps to reproduce:

1. Launch Instantbird (nightly trunk) and start a new conversation.
2. Launch Accerciser[*]
3. Monitor object events in Accerciser by doing the following [**]:
   a. Select Instantbird in the list of apps
   b. Switch to the Event Monitor page
   c. Select the 'Selected application radio button
   d. Check the checkbox for 'object'
4. Wait until your chat partner changes status (e.g. starts typing)

Expected results: Some (reasonable/logical) event would be emitted by Instantbird which would distinctly identify to assistive technologies that the status had changed.

Actual results: No events (not even silly, irrelevant, or just-plain-wrong ones) are emitted. Therefore, ATs have no way of knowing that the status had changed.

5. Examine the attributes of the page tab for the chat partner: [***]
   a. Locate and select the page tab on the left
   b. Switch to the Interface Viewer page
   c. Expand 'Accessible'
   d. Look at the list of Attributes
   
Expected results: There would be an attribute which reflected the status (typing, stopped typing, away)

Actual results: There is no attribute reflecting the status. As a result, even if we had the event requested above (or a suspicion that the status might have changed), there's no way to find out what the status changed to. We need this information in order to be able to identify and present only changes of interest to the user (e.g. just changes w.r.t. typing).

* http://live.gnome.org/Accerciser
** See screenshot 1 if you are unfamiliar with Accerciser
*** See screenshot 2 if you are unfamiliar with Accerciser

===============================

Observations/Suggestions/Etc.:

1. I would've expected an object:text-changed:{insert,delete} event each time text is {added,removed from} the status bar. HOWEVER, from the perspective of an AT, text changing in the statusbar is not the best way to identify a status change. And yes, I know how silly that sounds. :-) But an awful lot of applications have a statusbar and quite a few of those apps frequently change the text in their statusbar. Therefore, an AT such as the *nix screen reader Orca tends to ignore events from a statusbar. (Users have a command in Orca by which to read the text in a statusbar on demand should they have a want/need to do so.)

Something that most chat apps (including Instantbird) do when a chat partner's status changes is change the appearance of the page tab. Therefore, what would be awesome is a logical, relevant event emitted by the accessible associated with the page tab whose appearance changed. The two candidates (to me) would be object:visible-data-changed and object:text-attributes-changed. Because it appears that AtkText has not been implemented for this widget, my vote would be for object:visible-data-changed.

2. There are no official rules as to what the name and value of an AtkObject attribute must be. Therefore, to a certain extent, you should feel free to roll your own. Anything in the ballpark of

   {buddy-}status:{typing,stopped-typing,away}

should be fine. As long as it's predictable and easy for an AT to identify....

3. If you have any questions about the above, I'd be happy to try to answer them. Alternatively, since this appears to be a XUL widget, there's  #accessibility on the Mozilla IRC. A lot of those guys can probably help with any issues or questions you have. The ones doing a11y development on a regular-to-daily basis (AKA my "go-to guys") are:

  * Marco Zehe
  * David Bolter
  * Alexander Surkov
  * Ginn Chen

Finally/FWIW:

I recently added support for Instantbird to Orca at the request of one of our users. Often when we get such a request, what we find is that there are so many a11y issues with the app that it's almost not worth the trouble in trying. I found the very opposite with Instantbird: While there are some nits (some yours, some mine) to be worked out, the overall Instantbird experience for GNOME users who are blind seems quite nice. Thank you for developing such an accessible application! If we can just get the above resolved, life will be grand. :-)
(Reporter)

Comment 2

4 years ago
Created attachment 8352044 [details]
Screenshot 2: Accerciser interface viewer

*** Original post on bio 409 as attmnt 303 by joanmarie.diggs AT gmail.com at 2010-04-30 22:59:00 UTC was without comment, so any subsequent comment numbers will be shifted ***
*** Original post on bio 409 at 2010-05-04 07:59:40 UTC ***

First thank you for working on this, and for your nice and detailed bug report. We appreciate it, really.

I don't have a fix to propose yet, but I guess it's better for you if I give some update here :).

On the DOM node of each tab conversation tab, there's a "status" attribute which contain a value like "available", "away", "idle", ... When a buddy is typing, there's an additional attribute "typing" with the value "true", and when the buddy has stopped typing, the attribute "typing" is replaced by an attribute "typed" (with the value "true" too).

I spent some time during the week-end reading documentation related to the accessibility code in Mozilla but I haven't found how I can expose additional attributes on an accessible node.

I asked a question in #accessibility about this on Sunday, but it probably wasn't an appropriate time to get an answer.
Status: UNCONFIRMED → NEW
Ever confirmed: true
(Reporter)

Comment 4

4 years ago
*** Original post on bio 409 by Marco Zehe (:MarcoZ) <marco.zehe AT googlemail.com> at 2010-05-04 13:48:34 UTC ***

(In reply to comment #2)
> On the DOM node of each tab conversation tab, there's a "status" attribute
> which contain a value like "available", "away", "idle", ... When a buddy is
> typing, there's an additional attribute "typing" with the value "true", and
> when the buddy has stopped typing, the attribute "typing" is replaced by an
> attribute "typed" (with the value "true" too).

A good, albait a bit undocumented, way is to use WAI-ARIA to create additional attributes. If you use an attribute which WAI-ARIA doesn't know about, we will have a catch-all branch in Gecko which will unconditionally convert this unknown ARIA attribute to an accessible object attribute.

So on the DOM node, add another few attributes:

- aria-buddystatus with the value that you also assign to that other attribute.
- aria-buddytyping set to "true" if a buddy is typing.
- aria-buddytyped analogous.

Additionally, in Gecko 1.9.2 or later, if one of these object attributes changes, we'll even fire an event to indicate this (see Mozilla bug https://bugzilla.mozilla.org/show_bug.cgi?id=510441).

I hope this helps!
Created attachment 8352046 [details] [diff] [review]
Patch v1

*** Original post on bio 409 as attmnt 305 at 2010-05-04 21:26:00 UTC ***

(In reply to comment #3)

> A good, albait a bit undocumented, way is to use WAI-ARIA to create additional
> attributes. If you use an attribute which WAI-ARIA doesn't know about, we will
> have a catch-all branch in Gecko which will unconditionally convert this
> unknown ARIA attribute to an accessible object attribute.

This helped a lot, thanks! The attached patch uses this.

> Additionally, in Gecko 1.9.2 or later, if one of these object attributes
> changes, we'll even fire an event to indicate this (see Mozilla bug
> https://bugzilla.mozilla.org/show_bug.cgi?id=510441).

We use Gecko 1.9.2.
After adding the aria-<name> attributes, I haven't noticed any new accessible event when testing with accerciser or DOM Inspector.
I may have misread the patch in the bug you linked, but it seemed to me that this is only firing the event when the attribute which is changed is "aria-grabbed" or "aria-dropeffect".

If I need to do something to get an event fired, do you have any pointer to an example of how I can do it?

Once again, thanks for your help!
(Reporter)

Comment 6

4 years ago
*** Original post on bio 409 by Marco Zehe (:MarcoZ) <marco.zehe AT googlemail.com> at 2010-05-05 04:28:12 UTC ***

(In reply to comment #4)
> I may have misread the patch in the bug you linked, but it seemed to me that
> this is only firing the event when the attribute which is changed is
> "aria-grabbed" or "aria-dropeffect".

You are absolutely right! I didn't look at the patch again when I linked the bug yesterday, and it's been awhile...I thought we had made it more generic. :(

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=563862 for expanding the firing of the event to either all, or those object attributes that have been created by aria attributes unknown to us.
(Reporter)

Comment 7

4 years ago
*** Original post on bio 409 by Marco Zehe (:MarcoZ) <marco.zehe AT googlemail.com> at 2010-05-05 08:52:54 UTC ***

>+           this.tab.setAttribute("aria-" + aAttrName, aValue);

Suggestion: To be absolutely usre that none of these will conflict with future "real" ARIA attributes, add a unique app identifier such as "ib" (for Instantbird), like:

+           this.tab.setAttribute("aria-ib" + aAttrName, aValue);
(Reporter)

Comment 8

4 years ago
*** Original post on bio 409 by Peter Vagner <pvdeejay AT gmail.com> at 2010-05-05 18:59:55 UTC ***

Hello,
Thanks everybody for great support.

Here is another idea. Wouldn't it be better to add alerts displaying these event notifications? I imagine it'll work the same way like topic anouncements work in the group chat sessions.

Because if instantbird will expose custom states ATs will need to be aware of the change while alerts are automagically accessible.

Excuse if this is not really suitable perhaps I am not aware of all the details.
(Reporter)

Comment 9

4 years ago
*** Original post on bio 409 by Joanmarie <joanmarie.diggs AT gmail.com> at 2010-05-05 19:52:09 UTC ***

(In reply to comment #7)
> Hello,
> Thanks everybody for great support.
> 
> Here is another idea. Wouldn't it be better to add alerts displaying these
> event notifications? I imagine it'll work the same way like topic anouncements
> work in the group chat sessions.

Define alert. :-)

Orca (the screen reader for GNOME) has a feature for chat apps in which a user can optionally be informed when the person he/she is chatting with has started typing or has stopped typing. As a sighted user, I can obtain that information from the status bar. In addition, I can look at the page tab because the text in the page tab changes color.

Question is, how can we provide this information to users who are blind, but only if they want this information provided to them? And if they want this information provided to them, they won't want to have to dismiss anything, move focus back to where they were before, etc.

I say all of this because "alert" to me means an old-school message box/dialog box. If what you mean is doing some sort of ARIA goodness which enables Orca to do the right thing as described above, then I have no objections.

Thanks!
*** Original post on bio 409 at 2010-05-05 20:12:09 UTC ***

(In reply to comment #7)

> Wouldn't it be better to add alerts displaying these
> event notifications? I imagine it'll work the same way like topic anouncements
> work in the group chat sessions.

I think the 'alert' here is the notification in the notification bar at the top of conversations. That's what is used to display the topic in multi-user chats.

If this is what you mean, then it's not really appropriate for typing notifications because having such a notification at the top of the conversation appear and disappear repeatedly would be very disturbing for sighted users. These notification areas are very visible, typically with a different background color, and there's an animation when they appear or disappear.

Thanks for the idea anyway!
(Reporter)

Comment 11

4 years ago
*** Original post on bio 409 by Peter Vagner <pvdeejay AT gmail.com> at 2010-05-06 05:10:52 UTC ***

Hello,
Now I am not sure how to describe what I mean.
For example firefox tends to show alerts in the sidebar when requesting user's interaction e.g. the box saying do you want to remember this password?
Technically it can be any content and when given the role alert using wai aria screen readers should pick it automagically. E.G. gmail uses alerts ofor its notifications too.
(Reporter)

Comment 12

4 years ago
*** Original post on bio 409 by Peter Vagner <pvdeejay AT gmail.com> at 2011-11-12 18:43:24 UTC ***

Hello,
After a while and a lot of changes to the instantbird code I can observe the following.
While my chat partner is typing it's available as a readable text in the status bar.
So while doing a quick conversation I can check manually whether the chat partner is typing.
Additionally status changes (such as when a person goes offline or changes status to away) are now displayed in the output window.
As soon as we'll get back to the bug 953554 (bio 103) we may be able to make the screen readers to pick these automatically like with message events.
I think this sounds good.

Comment 13

4 years ago
*** Original post on bio 409 at 2012-04-25 15:24:03 UTC ***

(In reply to comment #5)
> I filed https://bugzilla.mozilla.org/show_bug.cgi?id=563862 for expanding the
> firing of the event to either all, or those object attributes that have been
> created by aria attributes unknown to us.

FYI https://bugzilla.mozilla.org/show_bug.cgi?id=563862 now fixed in moz15

Comment 14

4 years ago
*** Original post on bio 409 at 2012-09-01 23:23:46 UTC ***

Is this bug still current or have all these issues been fixed as per comment 11?

Comment 15

4 years ago
*** Original post on bio 409 at 2012-10-17 18:41:47 UTC ***

Resolving as WFM. Please do reopen if there are still unfixed issues - or even better, file new bug(s), as this one has become quite confusing.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.