1813492 - Screen reader accessibility of message list broken in daily build since 2023-01-19

Status: RESOLVED FIXED

(Thunderbird :: Folder and Message Lists, defect, P1)

Description

[Robert Murray]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/109.0

Steps to reproduce:

Yuse tab and arrows to navigate the message list, using NVDA screen reader

Actual results:

Since 2023-01-19, the message list behaves differently with screen readers. NVDA only reads the subject column.

Expected results:

All columns are read by NVDA, as they are in earlier versions.

Comment 1

[Ellie Lassince]

Confirmed in Daily, using Orca on Linux

Comment 2

[Wayne Mery (:wsmwk)]

Thanks for filing the bug report

Comment 4

[Peter Vágner]

There are multiple regressions while using this new message list with screen readers.

• When selecting multiple messages it's not communicated to screen readers,
• It is not possible to activate right click popup menu by pressing applications key or shift+F10
• It has too many tab stop elements in the taborder that should get changed to the toolbar pattern I believe.
• It does not convey collapsed / opened state for threaded conversations
• It's not intuitive to move focus from quick filter to the message list

These are all I have noticed and can explain so far.
Should I open new issue for each of these?

Comment 5

[Thomas D. (:thomas8)]

(In reply to Peter Vágner from comment #4)

Thanks Peter!

There are multiple regressions while using this new message list with screen readers.

• When selecting multiple messages it's not communicated to screen readers,

^^ Yes, please file this.

• It is not possible to activate right click popup menu by pressing applications key or shift+F10

^^ Context menu via keyboard should be fixed by Bug 1816741, about to land.

• It has too many tab stop elements in the taborder that should get changed to the toolbar pattern I believe.
• It does not convey collapsed / opened state for threaded conversations
• It's not intuitive to move focus from quick filter to the message list

These are all I have noticed and can explain so far.
Should I open new issue for each of these?

Yeah, I think that would be good. Please file separate bugs for each of these issues. Tia!

Comment 6

[Wayne Mery (:wsmwk)]

Let's ask Mel, who has a document which explores the accessibility issues.

Comment 7

[Wayne Mery (:wsmwk)]

c/Mel/Elizabeth/

Comment 8

[Peter Vágner]

With previous implementation I was used to hold down ctrl key and navigate over the list of folders with up and down arrow key. It worked so I was changing selection but not actually opening that folder. When I was next to the item I wished to open I released the ctrl key and pressed arrow key once more. I am not sure if this is still possible as when holding ctrl or shift key and navigating with arrow keys does not send no events when using a screen reader.
Should I create all these as seperate issues or should I wait a bit with it?

Comment 9

[Elizabeth Mitchell]

For context, I did an initial screen reader accessibility analysis last week focused on the message thread pane/message list and went over findings with our team last week. Thomas has been adding bugs into Bugzilla from our findings. We have been marking duplicates and related bugs/see also.

The initial bug report by Robert Murray is its own unique bug, as it relates to the NVDA screen reader.

Comment 10

[Elizabeth Mitchell]

(In reply to Peter Vágner from comment #8)

With previous implementation I was used to hold down ctrl key and navigate over the list of folders with up and down arrow key. It worked so I was changing selection but not actually opening that folder. When I was next to the item I wished to open I released the ctrl key and pressed arrow key once more. I am not sure if this is still possible as when holding ctrl or shift key and navigating with arrow keys does not send no events when using a screen reader.
Should I create all these as seperate issues or should I wait a bit with it?

Peter, please go ahead and create bugs for these issues. This includes the 3 not yet addressed bugs noted in comment #5. If you do a search and do not find the same bug, yes, please add each of them.

This new bug you are mentioning sounds like it's the folder pane, where the mail folders are. Is that right?

Comment 11

[Peter Vágner]

I have discovered the whole row gets reported when arrowing over a message with empty subject. So this may not be that difficult to fix I guess.

Comment 12

[Peter Vágner]

Today I have downloaded Thunderbird 112.0a1 20230302 and it appears navigating in the list is now inpossible with a screen reader. Pressing up and down arrow keys causes screen reader to report nothing.
It looks similar to bug 1819276

Comment 13

[Ali Savas]

I would like to add that often any message is opened when navigating through the message list with the arrow keys. I was able to reproduce this behavior with JAWS 2023.2302.15.

Comment 14

[Alessandro Castellani [:aleca]]

The way forward for this is to create a fluent string containing all the currently visible columns, and set that as aria label. https://searchfox.org/comm-central/rev/fa356613fc1cf593c52bb71074e02bf76267c0af/mail/base/content/about3Pane.js#3259

The string should read in a natural way, something like "Message from "sender", titled "subject", received on "date", with attachments, marked as spam", etc.
The string above it's just an example on how it should be generated based on visible columns.

Comment 16

[Peter Vágner]

I would say the string should not contain words that may potentially duplicate its role, nesting level, expanded state and selection state. That include words such as message, item and similar. These should be communicated through the other means.
The string should be composed of name and value pairs for header fields listing addresses and display names such as from / to / cc.
Subject value should be presented with no name (without the word subject) if the subject or any other value is empty it should not be presented.
Other items should either be presented in the string or omitted according to the message properties e.g. star, attachments, spam and unread. I think column header names might be used for most of them.
There might be exceptions when the value is containing the meaning of the name such as priority. It should not say priority low but low priority.
I don't have strong opinion on date vs received column. I think no one will enable both at once thus simply including the date value without saying received or date might be enough.
Read status and unread should work differently. Read status should say either read or unread according to the individual message and unread column should either say unread or nothing in case the message is read.
I am not sure about the status but I think it should just say the status value e.g. replied without the name.
Size and total should just say value with the unit e.g. 15 kB.

Comment 17

[Peter Vágner]

I haven't yet discovered when exactly this has started but now the message list is not useable to screen reader users at least on linux.
We are missing focus events when the selection changes e.g. when arrowing up and down in the list of messages.
What I can say that daily 20230227 is emitting these focus events and 20230302 and newer builds are not.
I would say this is major as I am unable to do further testing until this is addressed. Should I create a new bug or this one is enough?

Comment 18

[Elizabeth Mitchell]

(In reply to Peter Vágner from comment #17)

I haven't yet discovered when exactly this has started but now the message list is not useable to screen reader users at least on linux.
We are missing focus events when the selection changes e.g. when arrowing up and down in the list of messages.
What I can say that daily 20230227 is emitting these focus events and 20230302 and newer builds are not.
I would say this is major as I am unable to do further testing until this is addressed. Should I create a new bug or this one is enough?

Hi Peter. Please create a new bug. What you mentioned is a regression because it was working earlier. Thank you!

Comment 19

[Peter Vágner]

I have filled the bug 1820604.

I have again looked more into this.
I have discovered individual cells of a message list already have meaningfull accessibility labels such as:

• Message not starred
• New message indicator
• Message not marked as spam

and so on. Perhaps it would be great to reword these to be less verbose and don't mention word message. I think words such as Starred, new, spam would suffice.
And then let the parent node receive the focus and allow platform to calculate accessible name of the parent item from its children.
I guess that would be the most natural way on how to handle this as adding new features would be easy and wouldn't require changes to the accessibility support of the whole message list.

Comment 20

[Alessandro Castellani [:aleca]]

Grabbing this since Elizabeth is on sheriff duty, and I found the issue in bug 1820604, so I should be able to tackle this fairly quickly.

Comment 21

[Peter Vágner]

I assumed this should also include ability to report all the message list item columns which is not yet addressed. Or is there a follow up bug I should be looking at instead?

Comment 22

[Peter Vágner]

Oops, that's my bad I thought this bug has been closed by an accident. But it's still open.
Excuse me.

Comment 23

[Alessandro Castellani [:aleca]]

No worries, it happens.
But yes, fixing this will make all the currently visible columns exposed to screen readers.
Like for the other bug, I'll link a try run so you can test it

Comment 24

[Alessandro Castellani [:aleca]]

Attachment: Bug 1813492 - Fix message list a11y. r=#thunderbird-front-end-reviewers

Comment 25

[Alessandro Castellani [:aleca]]

Try run: https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=1a4f743205599de93bb3b9d162f0bcba889399ae

Comment 26

[Peter Vágner]

(In reply to Alessandro Castellani [:aleca] from comment #25)

Try run: https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=1a4f743205599de93bb3b9d162f0bcba889399ae

Huge thanks.
With this build the column presentation is great however reporting focus changes is not as reliable as it should be I think.
Sometimes towards end of the list orca reports nothing when navigating with up / down arrow keys, sometimes it's presenting items that are showing or hiding at the beginning / end of the visible portion of the list before the select item.
Also is there someone to ask if there would be a way to reword some of the texts to be less verbose without the word message?

Comment 27

[Peter Vágner]

Hello,
Looking more into this I assume this patch is not the only change that affects my results.
As the list is now reported as having multi select capabilities to assistive tools.
When writing my previous comment I assumed screen reader complexity may affect it too but I can reproduce the fact what's reported as having the focus does not correspond with the message getting opened on the enter key press even with very simple focus listener that is completelly seperate from orca screen reader.
Are there other related bugs being worked on I might have overlooked?

Comment 28

[Alessandro Castellani [:aleca]]

(In reply to Peter Vágner from comment #26)

Sometimes towards end of the list orca reports nothing when navigating with up / down arrow keys, sometimes it's presenting items that are showing or hiding at the beginning / end of the visible portion of the list before the select item.

We're aware of this navigation issues, and we're tackling them in other bugs

Also is there someone to ask if there would be a way to reword some of the texts to be less verbose without the word message?

Sure, please open a bug with the strings you'd like to improve, and you can CC me on that.

(In reply to Peter Vágner from comment #27)

I can reproduce the fact what's reported as having the focus does not correspond with the message getting opened on the enter key press.

I can't reproduce this issue.
Please open another bug dedicated to this specific issue.

Comment 29

[Peter Vágner]

(In reply to Alessandro Castellani [:aleca] from comment #28)

(In reply to Peter Vágner from comment #26)

Sometimes towards end of the list orca reports nothing when navigating with up / down arrow keys, sometimes it's presenting items that are showing or hiding at the beginning / end of the visible portion of the list before the select item.

We're aware of this navigation issues, and we're tackling them in other bugs
Great I am testing with Thunderbird 112.0a1 20230312 and the navigation is now working fine.

I will file additional issues.

Comment 30

[tapper82]

Hi a11y is a mess when using NVDA on windows 11. This beta should of never bin shipped with such bad bugs.

Comment 31

[Ali Savas]

(In reply to tapper82 from comment #30)

Hi a11y is a mess when using NVDA on windows 11. This beta should of never bin shipped with such bad bugs.

The developers are aware that the message list needs to be reworked to make it more accessible again. I'm pretty sure the developers will do everything in their power to get most of it working again by version 115.

In your case it would have been very helpful to tell us what exactly is not working for you, so we can check it. Statements like "The message list is a mess" do not help. I recommend you please take a look at how to file bug reports. This includes:

• A description of the problem
• Exact steps to reproduce
• Actual result
• What you expect

I hope I could help you a little bit. If you have any questions, please write me privately.

Comment 32

[tapper82]

Folders like inbox do not say how many messages they have in like in the old build. The button that is labelled AppMenu button subMenu Display the Thunderbird Menu can be hidden because we can access the menus with the alt key.

Comment 33

[tapper82]

Wtf is Spaces Menu button subMenu 1 of 4

If you claps the folder tree view all the way by using the left arrow then you cant expand it again. you have to restart TB.

Comment 35

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/612cc1466bb3
Fix message list a11y. r=elizabeth

Comment 36

[Wayne Mery (:wsmwk)]

Comment on attachment 9322242 [details]
Bug 1813492 - Fix message list a11y. r=#thunderbird-front-end-reviewers

[Triage Comment]
Approved for beta

Comment 37

[Rob Lemley [:rjl]]

Thunderbird 112.0b5:
https://hg.mozilla.org/releases/comm-beta/rev/e8d0eb05bdef