1894098 - Content of the Select Translations panel is not announced to a screen reader user when the panel is opened

Status: VERIFIED FIXED

(Firefox :: Translations, defect, P3)

Description

[Anna Yeddi [:ayeddi]]

Attachment: Select Translations panel placed the focus to the middle of its content when opened and was announced just as "dialog" by NVDA

STR:

1. Ensure browser.translations.select.enable is set to true in the about:config
2. Ensure a screen reader is running
3. Navigate to a page in a language different from the OS, i.e. for English machine: data:text/html,<p lang="es_ES">Como estas?</p> or https://ru.wikipedia.org/wiki/%D0%A2%D0%B5%D0%BE%D1%80%D0%B8%D1%8F_%D0%BB%D0%BE%D0%B6%D0%B5%D0%BA
4. Select a text (using mouse or by following the NVDA 2024.1 guide for Native Selection Mode) and open a context menu/right click
5. Navigate to and activate Translate Selection to English
6. Confirm the dialog is opened and observe the screen reader announcements

Expected:

1. When a dialog is opened, it is expected that the focus would be placed to allow the user to discover the content of the dialog without the need to navigate backwards, which is usually for the content to be placed at the top of the content or on the container itself or to the main action but while making sure the content of the alertdialog would be announced by a screen reader.
   1. In this case, since the size of the dialog is short, and the primary action/target of the dialog is the translation, we could retain the focus on the <textarea> but it's important to use aria-describedby that is referring to the container of the dialog, to its content, so a screen reader user would become aware that there is, in fact, some content and controls that are placed above the current focus position.

Actual:

1. Only the dialog role itself is announced by screen readers, no information is provided about the settings or to/from language controls. Focus is placed on the translated output itself, which may cause users of assistive technology to think that there are no other content above, no way to toggle the to/from languages and/or change the translation settings.

Remediation:

1. To the role=alertdialog container, add aria-describedby reference to the ID attribute of the dialog itself - this would allow a screen reader to also read the content of the dialog to a user while the focus is placed not at the top of the dialog, without the need to reverse navigate it. Refer to the WAI ARIA alertdialog pattern

Comment 1

[Erik Nordin [:nordzilla]]

Attachment: Bug 1894098 - Announce SelectTranslationsPanel content to screen readers r=ayeddi

Ensures that the content of the SelectTranslationsPanel
is announced to screen readers when the panel is opened
by utilizing the aria-describedby attribute.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b037f347b38d
Announce SelectTranslationsPanel content to screen readers r=translations-reviewers,gregtatum

Comment 3

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/b037f347b38d

Comment 4

[Daniel Bodea [:danibodea]]

I am attempting to verify the fix in Nightly v128.0a1 from 2024-06-06 on Windows10+NVDA, MacOS11+VoiceOver and Ubuntu22+ORCA.

• The expected behavior of the Screen Reader when the panel is being opened is:
  "When a dialog is opened, it is expected that the focus would be placed to allow the user to discover the content of the dialog without the need to navigate backwards, which is usually for the content to be placed at the top of the content or on the container itself or to the main action but while making sure the content of the alertdialog would be announced by a screen reader.
  In this case, since the size of the dialog is short, and the primary action/target of the dialog is the translation, we could retain the focus on the <textarea> but it's important to use aria-describedby that is referring to the container of the dialog, to its content, so a screen reader user would become aware that there is, in fact, some content and controls that are placed above the current focus position."

• The behavior that is seen in the tested systems:

1. Ubuntu 22 + ORCA
   ** Focus -> The focus falls on the textarea and ORCA vocalizes: "main content, <reads first word of the selection> selected, read-only entry translating, focus mode, alert, translation complete". Then it awaits user interaction.
   ** Reading the text -> The focus is on the last row of the translated text, visually impossible to determine; If the user taps the Down-Arrow, the last row of the selection will be read. If he then taps Up-Arrow, the previous row is being read, and so on.
   ** Reading the UI -> Pressing TAB to move focus over the panel's elements results in ORCA reading the buttons and drop-downs efficiently, however, when the text area is selected again, it will only read one row at a time.
   In conclusion, the fix appears partial because the content of the alert dialog is not announced by ORCA, however, buttons, drop-downs and the textarea CAN be read if the user carefully navigates through them.

2. Windows 10 + NVDA
   ** Focus -> The focus falls on the textarea and NVDA vocalizes: "dialog, translation document, edit-read-only multiline, translating.
   ** After translation progress finishes -> NVDA vocalizes: "alert translation complete". Then it awaits user input.
   ** Reading the text -> The focus is on the last row of the translated text, visually impossible to determine; If the user taps the Down-Arrow, the last row of the selection will be read. If he then taps Up-Arrow, the previous row is being read, and so on.
   ** Reading the UI -> Pressing TAB to move focus over the panel elements results in NVDA reading the buttons and drop-downs correctly, however, when the text area is selected again, it will only read one row at a time.
   *** In conclusion, the fix appears partial because the content of the alert dialog is not announced by ORCA automatically, however, buttons, drop-downs and the textarea CAN be read if the user carefully navigates through them.

3. MacOS 11 + VoiceOver
   ** Focus -> The focus falls on the textarea and vocalizes: "<reads a short part of the selection>>, insertion on ...<interruption>,
   ** After translation progress finishes -> Voice Over vocalizes: "Firefox has new window, selection replaced.
   ** Reading the text -> If the user taps the Down-Arrow, VoiceOver will read the entire paragraph of translated text. The same thing happens if the user taps Up-Arrow.
   ** Reading the UI -> Pressing TAB to move focus over the panel elements results in VoiceOver reading the buttons and drop-downs correctly. When reaching focus over the text area, VoiceOver reads the whole paragraph.
   *** In conclusion, the fix appears partial because the content of the alert dialog is not announced by VoiceOver automatically, however, buttons, drop-downs and the textarea CAN be read if the user carefully navigates through them.

As a general conclusion, the latest build does not seem to respect the expected behavior described in comment 0. Could you give us some details about what the behavior should be, after the fix? I imagine that all the buttons and dropdowns should be presented in some way when the panel is opened, but the screen readers get interrupted by the appearance of the translated text inside the text area. Do you think this is what happens? How do you think we should address this? Thanks!

Comment 5

[Erik Nordin [:nordzilla]]

Daniel! Thank you for your thorough analysis and descriptions of the new behaviors here.

I'm going to have to ask Anna's opinion about the new flow of events here, and if she has any further recommendations.

I'm committed to continuing to get this right, even if we have to re-open this bug and uplift a subsequent patch into Firefox 128 beta.

---

Note: I am about to land a fix for Bug 1894233, which will at least ensure that the cursor starts at the beginning of the text, and should fix the parts of the above descriptions where they mention "…The focus is on the last row of the translated text…"

Comment 6

[Anna Yeddi [:ayeddi]]

Hi team, apologies for the long time taken to reply - and thank you both for the through investigation and really caring about the user's experience! <3

I filed a separate bug 1901314 to address the main outstanding issues that Daniel has mentioned in the comment 4 (excluding the arrow scroll that seems to be addressed by Erik in bug 1894233 per comment 5). I added a screenshot with the NVDA output with the patch submitted to the bug in the first comment in there - please do let me know if you think more information is needed. I also included a note in the expected behavior in the patch about VoiceOver - I suspect VO not announcing the label but only the description is a VO and/or platform bug and I'll investigate it separately when the patch lands.

Comment 7

[Daniel Bodea [:danibodea]]

I have verified the improvements implemented in bug 1901314:

1. NVDA and VoiceOver mention the label "beta" along with the accessible name "Translation".
2. NVDA, ORCA and VoiceOver announce the origin language and destination language set in the panel.

While these fixes DO slightly improve the situation, these fixes are pushed to Nightly129, but not to the Beta128.

While these improvements are very welcome to the implementation, they do not seem fix the main issue with this report, namely:

1. the content of the translation panel to be announced by the screen readers by default;
   OR
2. "In this case, since the size of the dialog is short, and the primary action/target of the dialog is the translation, we could retain the focus on the <textarea> but it's important to use aria-describedby that is referring to the container of the dialog, to its content, so a screen reader user would become aware that there is, in fact, some content and controls that are placed above the current focus position."

@Anna: Maybe I am not understanding the situation correctly. Can you help me understand? It seems that situation number 1 is not the chosen solution here, but then, what is the expected result? How do I verify that "a screen reader user would become aware that there is, in fact, some content and controls that are placed above the current focus position."?
Apologies for the nubie questions and regards!

Comment 8

[Anna Yeddi [:ayeddi]]

(In reply to Daniel Bodea [:danibodea] from comment #7)

I have verified the improvements implemented in bug 1901314:

1. NVDA and VoiceOver mention the label "beta" along with the accessible name "Translation".
2. NVDA, ORCA and VoiceOver announce the origin language and destination language set in the panel.

While these fixes DO slightly improve the situation, these fixes are pushed to Nightly129, but not to the Beta128.

While these improvements are very welcome to the implementation, they do not seem fix the main issue with this report, namely:

1. the content of the translation panel to be announced by the screen readers by default;
   OR
2. "In this case, since the size of the dialog is short, and the primary action/target of the dialog is the translation, we could retain the focus on the <textarea> but it's important to use aria-describedby that is referring to the container of the dialog, to its content, so a screen reader user would become aware that there is, in fact, some content and controls that are placed above the current focus position."

@Anna: Maybe I am not understanding the situation correctly. Can you help me understand? It seems that situation number 1 is not the chosen solution here, but then, what is the expected result? How do I verify that "a screen reader user would become aware that there is, in fact, some content and controls that are placed above the current focus position."?
Apologies for the nubie questions and regards!

Hi Daniel, no need to apologies - those are great and helpful questions and this situation is shown to be very unclear - with the panel labels being not announced by ORCA (bug 1902665) and VoiceOver (bug 1902662) - thank you for filing these bugs! The same reason is likely causing the description (that since the bug 1901314 includes the Manage Settings button which is announced by NVDA).

We want to make sure that from the announcement of a screen reader, a user who cannot see the screen or who relies on the SR audio to understand the content (i.e. someone with dyslexia), would hear the content of the panel from its top to the focus position (including the focused element, of course).

So, it seems that the NVDAis announcing everything as expected:

1. panel label ("Translations" with "Beta" at the moment)
2. panel description ("manage settings" button)
3. language pair selected ("To" and "From" with values)
4. The textarea (screen readers would differ in how they announce it and if they read the value, etc)

This covers everything that is on the panel above the current focus placement (the textarea which is autofocused), thus this would provide all expected information to a screen reader user, so they won't need to navigate backwards to review the content, only forward (to buttons in the footer). If something that is provided on-screen is not announced, this is a bug.

It looks like both the VoiceOver and ORCA are missing the points 1 and 2 - the label and description. These are now tracked in the bug 1902662 and bug 1902665 (I'm adding them as blocking to this bug for tracking purposes too).

And ORCA is not announcing the "Beta" badge. From the code, this image should be programmatically labeled (this could be checked in the Accessibility Inspector of DevTools), thus it is likely a (new?) bug of the platform or/and the ORCA itself, similarly to the label bugs.

I hope it helps, but do let me know if I can share any further information. Thank you for working on de-tangling it all!

Comment 9

[Daniel Bodea [:danibodea]]

Thank you very much for the clarifications, Anna!
Based on the last comment, we can agree that the NVDA behavior observed is the one intended, while the remaining issues are to be addressed in bug 1902662 and bug 1902665. This being said, I will verify this bug as fixed in Nightly v129.0a1. Before verifying this in Beta128, the fix of bug 1901314 needs to land in Beta. We will have the fixed build Beta v128.0b5 available for testing on Thursday (RO time).

Comment 10

[Daniel Bodea [:danibodea]]

I can confirm that, since bug 1901314 has landed in the Beta channel, the same behavior observed in Nightly v129.0a1 is seen in Beta v128.0b5.
NVDA behaves as expected while ORCA and VoiceOver screen readers have follow-up bugs to address the remaining system-specific issues.