757381 - [AccessFu] Indicate whether a text entry field is multi line.

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[Marco Zehe (:MarcoZ)]

Currently, AccessFu doesn't differenciate between single line and multi line text entries. I think multiline should be hinted at so the user knows they can also use vertical navigation for line by line nav.

Comment 1

[Marco Zehe (:MarcoZ)]

Attachment: Proposed patch

Is this the right place to add this (and similar) rules?

Comment 2

[Eitan Isaacson [:eeejay]]

Comment on attachment 626004 [details] [diff] [review]
Proposed patch

Review of attachment 626004 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks! The one clarifying question I have is whether this is relevant to anything besides entry roles, if not then I don't think there is a reason for a formatable string, and the utterance should be generated from a special entry function, and not in the default one.

::: accessible/src/jsat/UtteranceGenerator.jsm
@@ +203,5 @@
> +          let stateStr = (state.value & Ci.nsIAccessibleStates.STATE_MULTI_LINE) ?
> +            'objMultiLine' : 'objSingleLine';
> +          desc = gStringBundle.formatStringFromName(stateStr, [desc], 1);
> +        }
> +

Could there be another role, besides an entry, that we would need to have this for? If not, I would add this in a function called "entry".

::: dom/locales/en-US/chrome/accessibility/AccessFu.properties
@@ +77,5 @@
>  objNotChecked  =       not checked %S
>  objExpanded    =       expanded %S
>  objCollapsed   =       collapsed %S
> +objMultiLine   =       multi line %S
> +objSingleLine  =       %S

Same thing, if this is only relevant for entry accessibles, I would simply add a property, something like:
entryMultiLine = multi line entry

Comment 3

[alexander :surkov (:asurkov)]

(In reply to Eitan Isaacson [:eeejay] from comment #2)

> Could there be another role, besides an entry, that we would need to have
> this for? If not, I would add this in a function called "entry".

practically no but it doesn't sound that you need to care about role. In general if you have multiline state then you can say multiline. If you want to have a function then you can name it "multiline".

> Same thing, if this is only relevant for entry accessibles, I would simply
> add a property, something like:
> entryMultiLine = multi line entry

entry can have different states like mutliline, invalid, required etc, maybe it makes sense to announce states and then append a role

Comment 4

[Eitan Isaacson [:eeejay]]

(In reply to alexander :surkov from comment #3)
> (In reply to Eitan Isaacson [:eeejay] from comment #2)
> 
> > Could there be another role, besides an entry, that we would need to have
> > this for? If not, I would add this in a function called "entry".
> 
> practically no but it doesn't sound that you need to care about role. In
> general if you have multiline state then you can say multiline. If you want
> to have a function then you can name it "multiline".
> 

The objectUtteranceFunctions automatically maps roles to special functions, where defaultFunc is the fallback. So naming a function "entry", would actually map to entry objects, naming it "multiline" will do nothing.

Theoretically, you might not want to have each kind of object that has a multiline state announce it. It may be redundant and uninteresting to the user unless it is an entry. Of course, if it is only entries that have this state, it would not matter.

> > Same thing, if this is only relevant for entry accessibles, I would simply
> > add a property, something like:
> > entryMultiLine = multi line entry
> 
> entry can have different states like mutliline, invalid, required etc, maybe
> it makes sense to announce states and then append a role

The question is whether we want to keep object descriptions grammatically correct, or simply chain all the relevant states. The latter is more scalable, but could give radically different results depending on locale since the translator has no control over it.

Comment 5

[alexander :surkov (:asurkov)]

(In reply to Eitan Isaacson [:eeejay] from comment #4)
> The objectUtteranceFunctions automatically maps roles to special functions,
> where defaultFunc is the fallback. So naming a function "entry", would
> actually map to entry objects, naming it "multiline" will do nothing.
> 
> Theoretically, you might not want to have each kind of object that has a
> multiline state announce it. It may be redundant and uninteresting to the
> user unless it is an entry. Of course, if it is only entries that have this
> state, it would not matter.

while the mutltiline case works but how would it work for universal states like invalid, required, disabled applicable to any form control? each role lists all possible states?

> The question is whether we want to keep object descriptions grammatically
> correct, or simply chain all the relevant states. The latter is more
> scalable, but could give radically different results depending on locale
> since the translator has no control over it.

yes, localization could be a problem, just wondering how you combine list of states. Say you have invalid textbox (it should say: "invalid entry") and invalid textarea (should say: "invalid multiline entry"). It seems that keeping multiline and entry together you introduce new role rather than you try to combine state and entry. It might be tricky to keep all bunches of role + state correct.

Comment 6

[Eitan Isaacson [:eeejay]]

(In reply to alexander :surkov from comment #5)
> (In reply to Eitan Isaacson [:eeejay] from comment #4)
> 
> > The question is whether we want to keep object descriptions grammatically
> > correct, or simply chain all the relevant states. The latter is more
> > scalable, but could give radically different results depending on locale
> > since the translator has no control over it.
> 
> yes, localization could be a problem, just wondering how you combine list of
> states. Say you have invalid textbox (it should say: "invalid entry") and
> invalid textarea (should say: "invalid multiline entry"). It seems that
> keeping multiline and entry together you introduce new role rather than you
> try to combine state and entry. It might be tricky to keep all bunches of
> role + state correct.

I sincerely asked that question. If we give up on gramatical structure then things could look a lot simpler. I might propose something for that soon. This is more realistic now than it was before, because utterances don't need to be strict pairs of strings. So each state could be another string in the utterance.

Comment 7

[David Bolter [:davidb] (NeedInfo me for attention)]

Needs user research IMO.

Comment 8

[alexander :surkov (:asurkov)]

(In reply to Eitan Isaacson [:eeejay] from comment #6)

> I sincerely asked that question. If we give up on gramatical structure then
> things could look a lot simpler.

it should be grammatical correct. I hoped we can do that from states, roles whatever else blocks, for example, if localization provides a template sentence and localization for each block and then we just insert those blocks into template.

Comment 9

[Eitan Isaacson [:eeejay]]

(In reply to David Bolter [:davidb] from comment #7)
> Needs user research IMO.

fwiw, NVDA seems to go with the arbitrary chaining method. And I suspect iOS does too. The interesting thing is that iOS says "dimmed, text field", and NVDA says "edit unavailable".

Comment 10

[David Bolter [:davidb] (NeedInfo me for attention)]

I probably don't really understand the grammatical correctness argument. Alexander are you suggesting we form natural language generation? I think this would be fraught with peril.

Again, I think we should do what is best for users. Do we know? My assumption is that important information should come first (with no added verbal sugar) and less important information should come last, but at the same time not wantonly breaking the potential benefits of habituation on state/role orderings. Note my assumption is that we should optimize for efficiency.

CC+ Jamie,Mick

Comment 11

[James Teh [:Jamie]]

Different screen readers order the state and role information differently. However, I don't think many (if any) place a huge focus on grammatical correctness. There are just way too many possibilities. (As David suggests, it'd probably require natural language generation.) Certainly, we don't with NVDA. There are a few hard-coded exceptions. Notably, we say "visited link", even though we normally speak the states last. That said, it's worth noting that some users have complained about the inability to configure this order in NVDA.

Comment 12

[Eitan Isaacson [:eeejay]]

(In reply to James Teh [:Jamie] from comment #11)
> Different screen readers order the state and role information differently.
> However, I don't think many (if any) place a huge focus on grammatical
> correctness. There are just way too many possibilities. (As David suggests,
> it'd probably require natural language generation.) Certainly, we don't with
> NVDA. There are a few hard-coded exceptions. Notably, we say "visited link",
> even though we normally speak the states last. That said, it's worth noting
> that some users have complained about the inability to configure this order
> in NVDA.

That makes life easier. I started work on something like this in bug 757954.

On a side note, I think multi line should not be reported as a regular state and should be part of the object/type role utterance since it is not a transient state, it describes the permanent role of an object.

Comment 13

[Marco Zehe (:MarcoZ)]

(In reply to Eitan Isaacson [:eeejay] from comment #12)
> On a side note, I think multi line should not be reported as a regular state
> and should be part of the object/type role utterance since it is not a
> transient state, it describes the permanent role of an object.

Interesting point: In accessibility APIs, there's only one role for entries, and depending on the EXT_STATE_MULTI_LINE, it is determined whether it's that or single line. In HTML, we have two different elements for these, making this completely unambiguous. I'll come up with a new patch incorporating this idea soon.

Comment 14

[Marco Zehe (:MarcoZ)]

Comment on attachment 626004 [details] [diff] [review]
Proposed patch

Obsoleting this patch since it's the wrong approach for this case.

Comment 15

[Marco Zehe (:MarcoZ)]

Attachment: Patch2

Personally, I absolutely hate to have to copy all of this stuff into a new func, it's basically defaultFunc with one less states check and a simplified assignment for the multiline string. But here it is.

Comment 16

[Eitan Isaacson [:eeejay]]

Comment on attachment 626780 [details] [diff] [review]
Patch2

Review of attachment 626780 [details] [diff] [review]:
-----------------------------------------------------------------

This patch should depend on the state stuff I am doing in bug 757954.

I am unsure if we should incorporate it in this patch or not. But we need to utter both the entry name and its value. There are two approaches:
1. Format it into a special entry utterance function like we have here. So you would get something like:
"required First Name entry, John"
2. Or introduce the subtree presentation like I suggested in bug 753986, it would read the child accessibles as well (in the case of an entry, its value is a child text leaf), so landing on an entry with a value would "just work", it would read out something like this:
"required entry, First Name, John"

::: dom/locales/en-US/chrome/accessibility/AccessFu.properties
@@ +76,5 @@
>  objChecked     =       checked %S
>  objNotChecked  =       not checked %S
>  objExpanded    =       expanded %S
>  objCollapsed   =       collapsed %S
> +objMultiLine   =       multi line %S

I think we could simply have two non-formattable localizable strings.
One for a single line entry (perhaps "entry" in english), and one for a multi-line entry ("multi line entry" is a conservative option, but it could be anything, even "composition box"). This gives translators the flexibility to call that kind of entry whatever makes sense in that language.

That is what I was saying earlier about a multiline entry being an entirely different beast, and should really be considered a role of its own. You echoed that with the HTML input/textarea distinction.

Comment 17

[Marco Zehe (:MarcoZ)]

Attachment: Take 3

Based on the new architecture. I introduce the string role of "text area" here, which is shorter than "multi line text entry" or the like.

Comment 18

[Eitan Isaacson [:eeejay]]

Comment on attachment 627232 [details] [diff] [review]
Take 3

Review of attachment 627232 [details] [diff] [review]:
-----------------------------------------------------------------

There should be a use in this function of _getLocalizedStates so that information such as required and invalid are relayed. Otherwise it is a regression. Look at how it is used in defaultFunc.

Comment 19

[Marco Zehe (:MarcoZ)]

Attachment: Take 4, with states reinstated

Comment 20

[Eitan Isaacson [:eeejay]]

Comment on attachment 627247 [details] [diff] [review]
Take 4, with states reinstated

Review of attachment 627247 [details] [diff] [review]:
-----------------------------------------------------------------

Looks, good. One tip below, take it or leave it.

::: accessible/src/jsat/UtteranceGenerator.jsm
@@ +204,5 @@
> +      let roleStr = (aStates.ext & Ci.nsIAccessibleStates.EXT_STATE_MULTI_LINE) ?
> +                    gStringBundle.GetStringFromName('textarea') :
> +                    this._getLocalizedRole(aRoleStr);
> +      if (roleStr)
> +        desc.push(roleStr);

roleStr should always be non-empty. I would perhaps phrase it like this:

desc.push(this._getLocalizedRole(
            (aStates.ext & Ci.nsIAccessibleStates.EXT_STATE_MULTI_LINE) ?
              'textarea' : 'entry'));

Comment 21

[Eitan Isaacson [:eeejay]]

http://hg.mozilla.org/integration/mozilla-inbound/rev/49859cdadbc4

Comment 22

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/49859cdadbc4