423409 - All user interaction script events should be exposed as accessible actions

Status: NEW

(Core :: Disability Access APIs, defect)

Description

[James Teh [:Jamie]]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5pre) Gecko/2008030607 Minefield/3.0b5pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5pre) Gecko/2008030607 Minefield/3.0b5pre

Firefox currently exposes an accessible action of "click" for HTML elements which have an onClick scripting event attribute. Thus, assistive technologies can both use this information to report that such elements are clickable to the user and allow the user to activate the element as if it had been clicked. However, there are several other script event attributes that relate to interaction with the user which are not exposed as accessible actions, such as onMouseDown, onMouseUp and onDblClick. Therefore, these cannot be reported or activated by assistive technologies. Although these events are very specific to input method and should probably be avoided unless specifically required, sites such as Google Calendar unfortunately use onMouseDown in place of onClick. This is becoming increasingly prevalent. These should therefore be exposed as accessible actions.

Some of these interactive script events require parameters in order to be useful; e.g. onKeyDown. In these cases, allowing the action to be fired may be difficult, as one must determine what to pass in event parameters. If an accessible action which cannot be fired is undesirable, perhaps an alternative method could be used to communicate the event, such as accessible attributes. The implementation certainly needs to be discussed.

Reproducible: Always

Comment 1

[alexander :surkov (:asurkov)]

Would be nice to get some start documentation on this. Marco since you had similar idea already then could you prepare this? Also how we will ensure we aren't unique who will support this?

Comment 2

[David Bolter [:davidb] (NeedInfo me for attention)]

Mass un-assigning bugs assigned to Aaron.

Comment 3

[David Bolter [:davidb] (NeedInfo me for attention)]

We need to consider exposing the existence of onMouse* as object attributes. Also we might want to expose accessibles for the existence of any of onclick, onMouseDown/Up, onDblClick. IIRC we only do this for onclick.

Comment 4

[Joanmarie Diggs]

Hi guys.

What about adding onMouseOver to the mix? I am working on implementing support for onMouseOvers in Orca. And if I do say so myself, the functionality ain't all that bad. <smile> The one problem is that the user must know -- or at least suspect -- that a given object has an associated onMouseOver. It would be cool if instead we could automatically present the presence of an associated onMouseOver to the user.

Comment 5

[alexander :surkov (:asurkov)]

1) We could easy add onmousedown/onmouseup since we can think about them as onclick. So if you call DoAction then we'll send mousedown, mouseup and click events so all registered event listeners will be triggered.
2) onblckclick, onmouseover, onmouseout - we could expose them but there is no way to execute event listeners registered for these events. Do you expect to expose actions for them?
3) onkeydown/onkeyup/onkeypress - we need to expose additional info about expected keys we should extract from JS listeners - it's really subject for discussion and it's hard to implement.

Again, these attributes can be considered as hint only until we extract information from JS listener. Following example demonstrates it.

<ul onmouseover="if (event.target.localName='li') doSomething();">
<li>item1</li>
<li>item2</li>
</ul>

Note, some sites prefer to register event listener on document element and events as object attributes aren't helpful at all in this case.

Comment 6

[James Teh [:Jamie]]

(In reply to comment #5)
> 1) We could easy add onmousedown/onmouseup since we can think about them as
> onclick. So if you call DoAction then we'll send mousedown, mouseup and click
> events so all registered event listeners will be triggered.
This would be extremely useful; e.g. in Google Calendar.

> 2) onblckclick, onmouseover, onmouseout - we could expose them but there is no
> way to execute event listeners registered for these events. Do you expect to
> expose actions for them?
Ideally, actions would be good, but if it can't be done, just knowing that they're present would be good. These ones would probably be better as object attributes in this case.

> 3) onkeydown/onkeyup/onkeypress - we need to expose additional info about
> expected keys we should extract from JS listeners - it's really subject for
> discussion and it's hard to implement.
The key events might be unrealistic. In addition (and having thought about it more), the fact that an object has a key listener probably isn't "visible" to most users anyway, so there's probably no need to expose this information. It is an implementation detail for the site and not relevant to the user.

> <ul onmouseover="if (event.target.localName='li') doSomething();">
> <li>item1</li>
> <li>item2</li>
> </ul>
Ouch, yeah, that is nasty. Extracting this sort of info from the listener is unrealistic imo.

> Note, some sites prefer to register event listener on document element and
> events as object attributes aren't helpful at all in this case.
True enough.

I guess the one case I really want to see is onMouseDown. Perhaps this bug should be morphed accordingly?

Comment 7

[Marco Zehe (:MarcoZ)]

Note to self and Surkov: Fix bug 543796 by removing test_role_tablecell.html.

Comment 8

[alexander :surkov (:asurkov)]

(In reply to comment #6)

> I guess the one case I really want to see is onMouseDown. Perhaps this bug
> should be morphed accordingly?

I would say so. However I like to get feedback from Joanmarie sine she has concerns to mouseover (comment #4). 

We could file another bug for mouseover though and morph this one to mouseup/mousedown stuffs.

Comment 9

[James Teh [:Jamie]]

(In reply to comment #8)
> (In reply to comment #6)
> 
> > I guess the one case I really want to see is onMouseDown. Perhaps this bug
> > should be morphed accordingly?
> I would say so. However I like to get feedback from Joanmarie sine she has
> concerns to mouseover (comment #4). 
To clarify, I'd like to see some exposure for mouseover. However, if it's not possible to do the action (fire the event with the centre coordinates of the object in question, perhaps?), then an object attribute would be good.

Comment 10

[alexander :surkov (:asurkov)]

(In reply to comment #9)
> (In reply to comment #8)
> > (In reply to comment #6)
> > 
> > > I guess the one case I really want to see is onMouseDown. Perhaps this bug
> > > should be morphed accordingly?
> > I would say so. However I like to get feedback from Joanmarie sine she has
> > concerns to mouseover (comment #4). 
> To clarify, I'd like to see some exposure for mouseover. However, if it's not
> possible to do the action (fire the event with the centre coordinates of the
> object in question, perhaps?), then an object attribute would be good.

Sorry I was unclear I think Gecko should have a way to send mouseover/mouseout events to the element. I meant we don't expose these listeners anyhow in a11y and we should introduce something new for this. It could be two additional actions. If it works for you then I could file follow up bug to deal with separately.

Comment 11

[alexander :surkov (:asurkov)]

(In reply to comment #10)

>If it works for you then I could file follow up bug to deal with
> separately.

or we could make this bug to a meta and file new bugs for every case we want to deal with. But it's not big deal.

Comment 12

[alexander :surkov (:asurkov)]

I filed bug 543931 for mouseup/down and bug 543932 for mouseover/out.