1613045 - No longer able to know who I write to (using a screen reader)

Status: RESOLVED FIXED

(Thunderbird :: Message Compose Window, defect, P1)

Description

[Jean-Philippe MENGUAL]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0

Steps to reproduce:

With a screen reader such as orca on:

1. Reply to a mail (ctrl-r or reply to)
2. shift-tab until the fields where destination addresses are mentioned

So far, the caret reached the addresses and spoke their status (to, Cc etc). Even in the new layout of the window. Now, shift-tab takes the caret on the labels, which should be buttons (cf bug 1611793) but no longer on any address. Impossible then to review the addresses you reply to or you are writing to.

Actual results:

The caret should be able to go on the addresses, expose them to the screen reader, and they should be removable via del key.

Comment 1

[Alex ARNAUD]

I confirm that too. Thanks for reporting this.

Comment 2

[Alessandro Castellani [:aleca]]

HI, thanks for reporting this.
This change was introduced in bug 1602372 to limit the amount of steps necessary to browse through all the recipient fields.
Once the focus ring is on a specific field, you can cycle through all the addresses with the left arrow key.

I can understand how this might not be intuitive, especially when using a screen reader since you won't know if a field has any addresses before trying moving the focus with the arrow key.

Maybe I could dynamically update the aria-label of the input field to communicate the amount of addresses currently present, so when the screen reader focuses on that field, it would read something like "To input field with 3 total addresses", or something like that.

Comment 3

[Richard Marti (:Paenglab)]

Alessandro, you could give the labels that are used as buttons the role="button". Then the screen readers should know the function of them.

Comment 4

[Alessandro Castellani [:aleca]]

(In reply to Richard Marti (:Paenglab) from comment #3)

Alessandro, you could give the labels that are used as buttons the role="button". Then the screen readers should know the function of them.

Sure, I'll do that in bug 1611793

Comment 5

[Alessandro Castellani [:aleca]]

Attachment: 1613045-pills-aria-label.diff

All right, I think this might be a viable solution for accessibility and screen readers.
Basically, the aria-label attribute of the addressing field will update accordingly to reflect the amount of pills associated with that field. The 3 possible variations are:

• { $type } input field
• { $type } input field with 1 recipient pill
• { $type } input field with { $count } recipient pills

Since the pills are not accessible via TAB focus anymore to prevent an infinite amount of tabbing before reaching the various input fields, updating the aria-label will reflect the current status of the addressing row, and by using the arrow keys, it's possible to cycle through all the pills on each addressing row.

Comment 6

[Alessandro Castellani [:aleca]]

Attachment: 1613045-pills-aria-label.diff

Sorry, I forgot to hg add the newly created fluent file.

Comment 7

[Magnus Melin [:mkmelin]]

Comment on attachment 9124611 [details] [diff] [review]
1613045-pills-aria-label.diff

Review of attachment 9124611 [details] [diff] [review]:
-----------------------------------------------------------------

Just starting up and adding two addresses, it seem after that I have a aria-labelledby="null" on the input

::: mail/locales/en-US/messenger/messengercompose/messengercompose.ftl
@@ +7,5 @@
> +#   $type (String) - the type of the addressing row
> +remove-address-row-type = Remove the { $type } field
> +
> +#   $type (String) - the type of the addressing row
> +#   $count (Number) - the of recipient pills currently present in the addressing row

the number of

@@ +9,5 @@
> +
> +#   $type (String) - the type of the addressing row
> +#   $count (Number) - the of recipient pills currently present in the addressing row
> +address-input-type = { $count ->
> +    [0]     { $type } input field

Empty { $type } input field

@@ +10,5 @@
> +#   $type (String) - the type of the addressing row
> +#   $count (Number) - the of recipient pills currently present in the addressing row
> +address-input-type = { $count ->
> +    [0]     { $type } input field
> +    [one]   { $type } input field with 1 recipient pill

I think "one" should be written out as text. I don't think we want to talk about pills here. Maybe
   [one]   { $type } input field with one recipient

@@ +11,5 @@
> +#   $count (Number) - the of recipient pills currently present in the addressing row
> +address-input-type = { $count ->
> +    [0]     { $type } input field
> +    [one]   { $type } input field with 1 recipient pill
> +    *[other] { $type } input field with { $count } recipient pills

here too, remove the "pills"

Comment 8

[Magnus Melin [:mkmelin]]

Maybe you could add a test for this somewhere, so we don't regress

Comment 9

[Alex ARNAUD]

After discussing with Jean-Philippe I think we should add up/down arrow to cycle through the addresses. In other application it's usually up/down arrow that they are used for this. Also to be complete for a blind person there is no complete representation of what it is drawn on the screen so learning to the users I train that they've to use left/right for just Thunderbird will create IMHO confusion.

We could indeed let left/right as it is now for sighted people and just add up/down.

Comment 10

[Thomas D. (:thomas8)]

(In reply to Alex ARNAUD from comment #9)

After discussing with Jean-Philippe I think we should add up/down arrow to cycle through the addresses. In other application it's usually up/down arrow that they are used for this. Also to be complete for a blind person there is no complete representation of what it is drawn on the screen so learning to the users I train that they've to use left/right for just Thunderbird will create IMHO confusion.

We could indeed let left/right as it is now for sighted people and just add up/down.

Alex ARNAUD, thank your for your input, which is welcome and apprecicated. However, I think in this case we'll make life harder for both types of people, sighted and blind, if we start to implement navigation which does not match the actual arrangement of items on the screen. Can you share the names of email applications where up/down navigates recipients horizontally?

Consider the following scenario: A message with 100 BCC-recipients. So on screen, in the rectangular BCC input field, we'll have something like 20 rows with 5 recipients each. You want to change the 51st recipient in row 11. In that case, we certainly want to offer cursor up/down for fast vertical navigation between the 20 rows, because lacking that, everyone would have to navigate through 50 recipients to get to any recipient in row 11. So if there's no fast vertical navigation, even for the blind, this would force you to go through and listen to the names of all 50 recipients to find the one you're looking for, which is inefficient and inconvenient. Maybe your recipients are alphabetic, so you might have an idea that Martha can't be at the beginning or the end. Maybe you just know from experience that Martha is somewhere in the middle.

So I would actually turn your argument upside down and say that especially for a blind person, it should be helpful and more efficient to get a correct representation of what is on the screen, namely that recipients are primarily horizontally aligned, so when they are more, they end up in rows, and we should allow everyone to navigate columns and rows of this recipient grid efficiently. Depending on scenario, if your memory is good, and your recipients stay the same, you can even develop patterns where you'll know that Martha is row 11 (that's 10 times cursor down), then the third recipient in that row (so that's 2 times cursor left, column 3, so to speak). So you can develop a mental grid which will be a correct representation of what is on screen. Which will also be helpful if you need to talk about it to the sighted, both for reporting or receiving information, or even getting assistance, because any grid references like "20th recipient going down" will now become meaningless if we allow horizontal navigation with vertical keys. So imho, your proposal would actually increase confusion and prevent efficient navigation for both the sighted and the blind.

Btw, as a blind person you might have wondered why we're always talking of pills on these bugs. Each recipient is a unit of display name and email address. We call them recipient pills because each recipient has a shaded background color with rounded corners, so that looks a bit similar to the 2D image of a medical pill.

Comment 11

[Alessandro Castellani [:aleca]]

Just starting up and adding two addresses, it seem after that I have a aria-labelledby="null" on the input

Uh, that's strange because I removed all the aria-labelledby attributes and not touching those anymore. Did it happen with the To field?

Maybe you could add a test for this somewhere, so we don't regress

Indeed. I will add tests once this implementation gets approved.

After discussing with Jean-Philippe I think we should add up/down arrow to cycle through the addresses.

As Thomas said, that might be confusing based on the order of the pills.
There's a filed bug I will work on later which will implement the ability of quickly reorder all the addresses (eg. alphabetically), and force an ordered grid with only 1 pill per row. At that point, the up/down arrow navigation might make sense.

Comment 12

[Alessandro Castellani [:aleca]]

Attachment: 1613045-pills-aria-label.diff

Strings updated, alongside a small fix to update the aria label when pills are dragged on the recipient field from the address book.

Comment 13

[Alex ARNAUD]

Alex ARNAUD, thank your for your input, which is welcome and apprecicated. However, I think in this case we'll make life harder for both types of people, sighted and blind, if we start to implement navigation which does not match the actual arrangement of items on the screen. Can you share the names of email applications where up/down navigates recipients horizontally?

Indeed, this makes sense to me.

There's a filed bug I will work on later which will implement the ability of quickly reorder all the addresses (eg. alphabetically), and force an ordered grid with only 1 pill per row. At that point, the up/down arrow navigation might make sense.

What issue are you talking about?

Comment 14

[Magnus Melin [:mkmelin]]

(In reply to Alessandro Castellani (:aleca) from comment #11)

Just starting up and adding two addresses, it seem after that I have a aria-labelledby="null" on the input

Uh, that's strange because I removed all the aria-labelledby attributes and not touching those anymore. Did it happen with the To field?

Figured it out, kind of. It was the hidden input field of the previous pill. Seems like a bug, but not this bug.

Comment 15

[Magnus Melin [:mkmelin]]

Comment on attachment 9124875 [details] [diff] [review]
1613045-pills-aria-label.diff

Review of attachment 9124875 [details] [diff] [review]:
-----------------------------------------------------------------

::: mail/components/compose/content/MsgComposeCommands.js
@@ +3540,5 @@
> +
> +/**
> + * Update the aria-label of the autocomplete input field.
> + *
> + * @param {XULElement} row - the recipient address-row.

I think we can keep it as just Element, it won't be xul for that much longer. Also, capital T.

::: mail/locales/en-US/messenger/messengercompose/messengercompose.ftl
@@ +10,5 @@
> +#   $type (String) - the type of the addressing row
> +#   $count (Number) - the number of recipient pills currently present in the addressing row
> +address-input-type = { $count ->
> +    [0]     Empty { $type } input field
> +    [one]   { $type } input field with one recipient

address

@@ +11,5 @@
> +#   $count (Number) - the number of recipient pills currently present in the addressing row
> +address-input-type = { $count ->
> +    [0]     Empty { $type } input field
> +    [one]   { $type } input field with one recipient
> +    *[other] { $type } input field with { $count } recipients

addresses

Comment 16

[Alessandro Castellani [:aleca]]

Attachment: 1613045-pills-aria-label.diff

Comment 17

[Alessandro Castellani [:aleca]]

Here's a successful try run, just for the record: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=d22c2350330334eaab6b521d96285ce3dc919db1

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/eb963eb6aec6
Update aria-label of addressing input fields to communicate to screen readers the amount of recipient-pills on focus. r=mkmelin

Comment 19

[Alessandro Castellani [:aleca]]

Comment on attachment 9125108 [details] [diff] [review]
1613045-pills-aria-label.diff

.

Comment 20

[Magnus Melin [:mkmelin]]

Comment on attachment 9125108 [details] [diff] [review]
1613045-pills-aria-label.diff

Nightly moves to beta on monday, so there are no more betas this cycle.

Comment 21

[Alessandro Castellani [:aleca]]

(In reply to Alex ARNAUD from comment #13)

There's a filed bug I will work on later which will implement the ability of quickly reorder all the addresses (eg. alphabetically), and force an ordered grid with only 1 pill per row. At that point, the up/down arrow navigation might make sense.

What issue are you talking about?

Bug 1603051, still needs further discussion and a proper action plan.