1894091 - A screen reader does not announce when a translation is added on the Select Translations panel

Status: VERIFIED FIXED

(Firefox :: Translations, defect, P3)

Description

[Anna Yeddi [:ayeddi]]

Attachment: Select Translations panel is already showing the translated text while NVDA has not announced anything

STR:

1. Ensure browser.translations.select.enable is set to true in the about:config
2. Ensure an NVDA screen reader is running
3. Navigate to a page in a language different from the OS, i.e. for English machine: data:text/html,<p lang="es_ES">Como estas?</p> or https://ru.wikipedia.org/wiki/%D0%A2%D0%B5%D0%BE%D1%80%D0%B8%D1%8F_%D0%BB%D0%BE%D0%B6%D0%B5%D0%BA
4. Select a text (using mouse or by following the NVDA 2024.1 guide for Native Selection Mode) and open a context menu/right click
5. Navigate to and activate Translate Selection to English
6. Confirm the dialog is opened and observe the screen reader announcements:
   1. When the dialog is opened and the focus is placed on the <textarea[readonly]> while the translation is in progress and the placeholder Translating... is announced
   2. Once the translation is completed, observe the change in the announcement

Expected:

1. Once the translation is completed and the placeholder is replaced with the translated text, this translated text is announced by a screen reader.

Actual:

1. Once the translation is completed and the placeholder is replaced with the translated text, nothing is being announced by a screen reader. User will have to navigate away from the focused area and re-focus it to hear the translated text.

Note:

NVDA on Windows 11 did not announce the textarea change (the dialog is alertdialog) while VoiceOver on macOS Sonoma 14.4.1 did announce the change, as expected.

Comment 1

[Anna Yeddi [:ayeddi]]

Possible approach is to follow the example provided by :jteh - autofocus the <textarea> while having aria-live="polite" on it to have some indication from NVDA when the placeholder text is replaced:
data:text/html,<textarea autofocus id="ta" aria-live="polite">Loading...</textarea><script>ta.select(); setTimeout(() => ta.value = "hello", 2000);</script>
NVDA would say: selection removed

Comment 2

[Erik Nordin [:nordzilla]]

Hey Emilio,

Problem Summary

We're running into an issue where mutating the text in the <textarea> from Translating... to whatever the translated text is, is not picking up for screen readers. There's nothing tracking mutations to the text area for screen readers.

The solution that Anna posted just above, if I'm understanding correctly (see video in comment 3), would have the text within the <textarea> visually selected when it says Translating... and then when the translation comes through, the selection on the Translating... text would be removed, which would announce selection removed for NVDA and then read the new text to the screen reader.

The problem is that this feels a little bit hacky, forcing it to say "selection removed" like that. And it also has the visual downside of having the Translating... placeholder text to always be visually highlighted, which UX would like to avoid if possible.

A Possible Solution?

So, do you recall early on when I was implementing this, I was swapping out the <textarea> element with a <div> element to display the Translating... placeholder text because I couldn't figure out a clean way to do the spinner SVG animation inside the <textarea>? This was flagged in review because we couldn't really guarantee that every aspect of the <div> styling would look identical to the <textarea> styling, and we could potentially see flashes of different styles when they are swapped. We fixed this with the animated background-image spinner for the <textarea>, and now there is only one <textarea> element that updates its text between the Translating... placeholder and the translated text when it is ready.

I wonder if a better (though still fairly hacky) solution might be to go back to something similar to the first approach, but instead of swapping out a <div> and a <textarea>, we would swap out two identically styled <textarea> elements: one for the placeholder text with the loading spinner background, and one for the translated text. I feel like since these are both <textarea> elements we would avoid the mismatched styling issue, and if they're both aria-live="polite" then it should announce it to screen readers whenever one gets hidden and the other becomes visible, right?

---

Other than that, I'd be open to brainstorming further ideas. This is an A11y S2 bug, and needs to be solved before release, but I'd like to try to find a way to do it without announcing an unnecessary "selection removed," and without adding any extra visual highlighting of the placeholder text, if possible.

I wish there was some sort of way to use a mutation observer and trigger the screen reader to start reading or something, but it doesn't sound like that API exists.

Comment 3

[Erik Nordin [:nordzilla]]

Attachment: anna-solution.webm

Here is a video of what I think Anna's solution is describing, which visually highlights the Translating... placeholder text, which we would like to avoid if possible while still solving the A11y defect.

Comment 4

[Emilio Cobos Álvarez (:emilio)]

I'm not sure I follow. Doesn't this reproduce on regular web content too? If so, it seems this should be fixed in the accessibility engine, right? Or am I missing something? What prevents web content from hitting the same issue?

Comment 5

[James Teh [:Jamie]]

It can't be fixed in the accessibility engine. There isn't a way to force a screen reader to read text added to a textarea other than by selecting it.

aria-live="polite" has no effect for textareas with NVDA, since it tends to result in a lot of annoying verbosity; e.g. when the user types or pastes into the field.

The replacement textarea hack should work. But honestly, I wonder whether the easiest solution here is just to A11yUtils.announce a message like "Translation complete" or similar.

As a side note, I'd argue it's an odd UX to have a status message inside a textarea in the first place. A textarea is for navigable text. But that's probably not an argument I'm going to win. :)

Comment 6

[Erik Nordin [:nordzilla]]

So is it okay with everyone if I try out the <textarea> swapping hack?

I think that would provide a seamless experience for screen readers to announce the new text without having it say "selection removed," while also maintaining the current visual appearance of not having any placeholder text highlighted.

I don't want to start on it if it won't be approved, or if everyone would prefer to keep looking for other alternatives.

---

Morgan, I'm NI'ing you on this too because I know you have good working knowledge of layout code and accessibility code, and I'd value your perspective about the <textarea> swapping solution described in the comments above.

Comment 7

[Emilio Cobos Álvarez (:emilio)]

I'd prefer not to add a very ugly workaround for this, which would break when somebody cleans it up :)

Can we do what Jamie suggested in comment 5?

Comment 8

[Erik Nordin [:nordzilla]]

I personally don't feel that it's that hacky. There is already logic to show/hide different buttons, message bars, and even the <textarea> itself depending on error states or success states that occur within the panel. Modifying it to swap out two mutually exclusive <textarea> elements would be only a few lines of code as well as being easily testable in automation.

Someone cleaning it up later would break my own test cases before they realized they broke an a11y-aspect out in the wild. Plus, I'd never implement something unusual like this without extra clear documentation as to why it was implemented that way.

Nonetheless, I do understand if people don't want that kind of logic in the code, even if it seems justified to me based on the end user experience it would provide.

---

If I'm understanding correctly, the flow that Jamie described would be to announce "Translation complete" and leave it up to the screen-reader user to navigate to the text area and select the text themselves?

If that is an experience that Jamie is okay with, then I am also okay with it.

Comment 9

[Emilio Cobos Álvarez (:emilio)]

The textarea has focus, right? So probably they don't need to move focus anywhere.

Comment 10

[Erik Nordin [:nordzilla]]

Right, the textarea has focus at this point.

Comment 11

[Morgan Reschenberg [:morgan]]

Yeah I agree with Jamie, I think that's the best path forward.

FWIW there are issues with a11yUtils.announce on macOS right now, but we're working on them :) If you rebase on the patch at D206083 things should work fine. If we go that route, lemme know and I'll make it a priority to get bug 1888310 resolved. I stopped working on it in the short term due to some finicky test failures (and we went another direction with the screenshots component which was the original blocker)

Comment 12

[Erik Nordin [:nordzilla]]

Thank you Emilio and Morgan for the input here!

Once I get final confirmation from Jamie that the proposal of using a11yUtils.announce is the direction we want to take after all of this discussion, then I believe we will have consensus and I will write up a patch for this tomorrow!

Comment 13

[James Teh [:Jamie]]

I think A11yUtils.announce is the most pragmatic, least bad solution here. I don't want to say it's perfect because it isn't, but I think swapping textareas is brittle as far as maintainability goes, as Emilio pointed out. Honestly, I think the best solution would be to not put the progress text in a textarea in the first place, but again, that's not a UX argument I'm going to win.

Comment 14

[James Teh [:Jamie]]

Hmm, I just saw you edited comment 8. Sorry, I was reading the original version in email.

If we put aside the code ugliness of the textarea swapping solution, it is probably a better experience for screen reader users because it will actually start reading the text, not just say "Translation complete". So I guess the trade-off here is ugliness/maintainability vs a slightly better user experience. At the end of the day, I could live with either solution.

Comment 15

[Erik Nordin [:nordzilla]]

Sorry Jamie, I feel like I'm often editing comments after submitting them to fix typos, to improve clarity, or to add something I forgot, and I should be more conscious about the fact that the first version that I submit is the version that shows up in people's emails.

---

Emilio, I understand that you would prefer not to swap the <textarea> elements, but I want to revisit this and ask whether you see this as just a code maintainability drawback, or if you think there would actually be some sort of functional problem with the solution itself.

I certainly recognize that the solution is unorthodox, but I feel confident that it would fit in relatively smoothly with the way the panel UI logic is implemented, and I feel confident in my ability to clearly document and test the behavior as the primary author and maintainer of this code for the foreseeable future.

I may be more hesitant to do something like this if it were a core piece of shared infrastructure that would expose an API surface to downstream code consumers, but this is an isolated behavior for a specific UI edge-case, and my opinion remains that I would rather take on the burden of implementing/documenting/maintaining something unusual if it provides a genuinely better user experience for everyone.

I'd be happy to tag you as a reviewer and also file a follow-up bug that is clearly linked in the code comments to investigate a more straightforward solution to announcing a <texarea> update to assistive technologies.

If you see this as a blocker for any sort of technical reason, I'd be very interested to know.
Otherwise, I'd like to reassert that I think this would provide the best user experience.

Comment 16

[Emilio Cobos Álvarez (:emilio)]

It is mostly a maintainability issue, but not sure in the way you're thinking...

My main concern is someone doing styling or some other changes not specific to this panel, and breaking this panel subtly because it does something so bespoke. This kind of thing happens somewhat easily with non-primary UI.

In general implementing such a hack should be a very last resort thing, IMO, and I'd still much rather avoid it.

Comment 17

[Erik Nordin [:nordzilla]]

Thank you for the input, Emilio.

I think that's a reasonable perspective, and given that Jamie has expressed, "at the end of the day, I could live with either solution," I'm inclined to follow your guidance here.

It would be a lot easier to justify shipping the feature with saying "Translation complete" and improving upon that behavior in the future, compared to implementing a hack to achieve the optimal experience on release with the worry of surprise regressions from seemingly out of nowhere.

I'm grateful for all of the discussion above, and I feel that the reasoning behind the decision is well documented.

I will be filing a follow-up bug to continue investigating a better way to announce the text automatically, and I will link to it in the comments next to the a11yUtils.announce call.

---

Morgan, given the final decision here to utilize a11yUtils.announce I would appreciate you putting Bug 1888310 back on your radar at your earliest convenience. And please let me know if there is anything I can do to help with that fix.

(Sending you a NI for the sake of notifying you about the decision. Feel free to clear it, no response needed.)

Comment 18

[Morgan Reschenberg [:morgan]]

Sounds good, will do :) thanks for the NI!

Comment 19

[Erik Nordin [:nordzilla]]

Attachment: Bug 1894091 - Announce translation complete via A11yUtils r=ayeddi

The SelectTranslationsPanel now uses A11yUtils to announce
to assistive technology when the translation has completed.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/29e2e1920d19
Announce translation complete via A11yUtils r=ayeddi,fluent-reviewers,translations-reviewers,gregtatum,bolsson

Comment 21

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/29e2e1920d19

Comment 22

[Daniel Bodea [:danibodea]]

Attachment: The initial presentation of the ST panel is confusing and more.webm

@Erik Nordin: This issue appears to have been fixed, but the problem still reproduces. This issue also seems to depend on bug 1888310. Does this mean the behavior will be fixed when bug 1888310 is also fixed?

The behavior observed in Nightly v128.0a1 on Windows10/11+NVDA is quite confusing: Firstly, NVDA wants to read something, but the translation process completes and interrupts it with the statement "Alert translation complete", then it verbalizes something I can't understand, then announces "Dialog, edit, read-only, multiline, translating". In other unclear situations, the screen reader might read the last or last few words from the textbox. The behavior is quite similarly confusing on all tested Screen Readers.

@Anna Yeddi: Could you explain what an acceptable screen reader output would be for this issue to be verified? Does the announcement "Alert translation complete" or "Selection replaced" suffice? how would you address this situation?

Comment 23

[Erik Nordin [:nordzilla]]

Hey Daniel,

Based on what Morgan said in comment 11 I think that this should probably work correctly on Windows and Linux, but not quite yet on macOS until Bug 1888310 is fixed.

However, it is fully tested and asserted that the correct calls are being made on macOS, so I expect everything to fall into place once Bug 1888310 lands.

It looks like Morgan was active on Bug 1888310 within the last day, and pushed up a new try run to see if issues were resolved, so my hope is that this will land before the feature is released.

Comment 24

[Anna Yeddi [:ayeddi]]

(In reply to Daniel Bodea [:danibodea] from comment #22)

@Anna Yeddi: Could you explain what an acceptable screen reader output would be for this issue to be verified? Does the announcement "Alert translation complete" or "Selection replaced" suffice? how would you address this situation?

Thank you for investigating the current behavior, Daniel! It is not the most delightful way it could work right now, but as you have mentioned, we do hope that the bug 1888310 would improve the behavior for macOS.

For Windows, either Alert: translation completed or Selection replaced would be a sufficient indication of the translation output change in the dialog. At least with what tools we have available now.

The injected text that is hard to discern is the active element on the web content page opened - whenever the programmatic focus was when the context menu was opened. It's the panel behavior that we have (the content's focused element or the website's title is announced by NVDA when a panel/dialog is opened). I hope this helps.

Comment 25

[Daniel Bodea [:danibodea]]

Based on my description of Screen Reader functionality in comment 22 and Anna's confirmation that the fix is sufficient even if the translation text isn't properly read afterward, I can confirm the fix of this report.

Each screen reader has a way to announce the fact that the translation process has been completed:
Windows+NVDA verbalizes: "Alert translation complete".
Ubuntu22+ORCAverbalizes: "Alert translation complete".
MacOS+VoiceOver currently verbalizes something similar to: "Selection replaced".

As mentioned in comment 23 and comment 24, the fix of bug 1888310 will supposedly improve the behavior on MacOS even further.
I will close this report as verified. Thank you.