1793419 - Screenreader does not read annotated text in PDF documents

Status: VERIFIED FIXED

(Firefox :: PDF Viewer, defect, P1)

Description

[Ardelean Oana, Desktop Test Engineering [:oardelean]]

Attachment: Screen Recording 2022-10-03 at 14.30.53.mov

Found in

• Firefox 106.0b7

Affected versions

• Firefox 106.0b7;
• Nightly 107.0a1;

Tested platforms

• macOS 12;
• Windows 10;
• Ubuntu 22;

Affected platforms

• macOS 12;
• Windows 10;
• Ubuntu 22;

Steps to reproduce

1. Launch Firefox and go to any pdf form, for example https://www.africau.edu/images/default/sample.pdf.
2. Click on the Text button from the upper right side of the PDF Toolbar.
3. Select a place on the PDF and enter any text.
4. Enable the screenreader(VoiceOver on macOS, NVDA on Windows, ORCA on Linux).
5. Focus the previously written text.

Expected result

• Screenreader should be able to read the text correctly.

Actual result

• Screenreader reads any text as "group"/"blank"(depending on OS).

Regression range

• Not a regressions since this is a new feature.

Additional notes

• I noticed that if the user is writing the text slowly with VoiceOver enabled, VoiceOver will read the input letter by letter. On Windows, NVDA reads "blank" instead of "group". Orca does not recognize the text at all on Linux if focused. Please see attached video with the issue.

Comment 1

[Morgan Reschenberg [:morgan]]

Re-triaging to PDF because I'm not convinced this is an a11y platform issue -- it sounds like we're exposing this element consistently across platforms ("blank" and "group" seem similar enough).

:marco, when a user adds annotated text, what kind of element do we use to display it? Is it an SVG?

Comment 2

[Calixte Denizet (:calixte)]

Attachment: image.png

:morgan, you can see in devtools how the annotation is structured (see the image in attachment).
I wonder if it could be related to the fact that the div we use to edit the annotation gets a contenteditable="true" when edited and this one will wrap each line in a div.

Comment 3

[Calixte Denizet (:calixte)]

I managed to have something working on Windows with NVDA in using the attribute aria-activedescendant to make it pointing to the nested div (the one containing the text).
I tested the patch on mac 12.6 with VoiceOver and a fresh Firefox build and unfortunately I'm still stuck on "You're in a group".
:morgan, would you have any advice ?

Comment 4

[Calixte Denizet (:calixte)]

With VoiceOver, once we're on a newly added text, it's possible for the reader to read the contents in hitting ctrl+option+right arrow.

Comment 5

[Calixte Denizet (:calixte)]

Attachment: pr 15595: [Editor] Make FreeText annotations visible for screen readers when in editing mode (bug 1793419)

Comment 6

[Morgan Reschenberg [:morgan]]

Followed up on github :) Let me know if you're still having that issue, things look good to me when I test this locally.

Comment 7

[Marco Castelluccio [:marco]]

It would be nice to uplift this, since this is a new feature in 106 and we want it to be as accessible as possible.

Comment 8

[Marco Castelluccio [:marco]]

[Tracking Requested - why for this release]: See comment 7.

Comment 9

[Donal Meehan [:dmeehan]]

:marco could you please confirm the bug tracking the pdf update which includes this fix?
Looks like Bug 1796243 needs a beta uplift request?

Comment 10

[Marco Castelluccio [:marco]]

Bug 1796243 contains the whole update, but we might want to cherry pick only the patch that fixes this bug specifically.

Comment 11

[Marco Castelluccio [:marco]]

Oana, could you verify the fix?

Comment 12

[Calixte Denizet (:calixte)]

Attachment: Bug 1793419 - Make FreeText annotations visible for screen readers in editing mode r=#pdfjs-reviewers

Comment 13

[Calixte Denizet (:calixte)]

Comment on attachment 9300146 [details]
Bug 1793419 - Make FreeText annotations visible for screen readers in editing mode r=#pdfjs-reviewers

Beta/Release Uplift Approval Request

• User impact if declined: Newly added annotations will be invisible for users using a screen reader.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: See the STR in comment #1.
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The patch is very small and it's just a matter of adding an aria property to a html element.
• String changes made/needed:
• Is Android affected?: No

Comment 14

[Ardelean Oana, Desktop Test Engineering [:oardelean]]

On Nightly 108.0a1(2022-10-25) Screenreader perceives the inserted text as follows:

• macOS 12(with VoiceOver): on focus, instead of “group”, it now says “Text Editor, group”. If navigating with CTRL+Option+Right Arrow, then VoiceOver will spell the correct words afterwards. Here's a recording for more detail on the exact behaviour.
• Windows 10(with NVDA): on focus, instead of “blank”, on the first tabbed word(which was test), the spelling is “Clickable test", but the rest of the inserted words are spelled correctly.
• Linux(with OS's native Screenreader): on focus, it now spells "Text Editor" before each inserted word. This applies to all words/phrases.

Comment 15

[Donal Meehan [:dmeehan]]

Comment on attachment 9300146 [details]
Bug 1793419 - Make FreeText annotations visible for screen readers in editing mode r=#pdfjs-reviewers

Approved for 107.0b6.

Comment 16

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/releases/mozilla-beta/rev/e65e06d19520

Comment 17

[Morgan Reschenberg [:morgan]]

(In reply to Ardelean Oana from comment #14)

On Nightly 108.0a1(2022-10-25) Screenreader perceives the inserted text as follows:

• macOS 12(with VoiceOver): on focus, instead of “group”, it now says “Text Editor, group”. If navigating with CTRL+Option+Right Arrow, then VoiceOver will spell the correct words afterwards. Here's a recording for more detail on the exact behaviour.
• Windows 10(with NVDA): on focus, instead of “blank”, on the first tabbed word(which was test), the spelling is “Clickable test", but the rest of the inserted words are spelled correctly.
• Linux(with OS's native Screenreader): on focus, it now spells "Text Editor" before each inserted word. This applies to all words/phrases.

:calixte, I think the VO and Windows behaviour here is acceptable, but the linux behaviour sounds like it makes annotations borderline unusable.
:eeejay is this an issue you've seen on linux before? having the role spelled out? I'm wondering if that's a platform bug vs. a markup one

Comment 18

[Eitan Isaacson [:eeejay]]

(In reply to Morgan Reschenberg [:morgan] from comment #17)

:eeejay is this an issue you've seen on linux before? having the role spelled out? I'm wondering if that's a platform bug vs. a markup one

I just tested it, and it seems to behave properly. Says role ("text editor") and the text contents of the field.

Comment 19

[Timea Zsoldos [:zstimi/tzsoldos], Desktop QA]

Tested on Windows 10, Ubuntu 20 and on macOS 12 on Firefox 111.0b5, I can confirm the behave of screen reader is correct.

Comment 20

[Marco Castelluccio [:marco]]

Needinfo not needed anymore, given the bug was fixed on Linux too according to comment 18 and comment 19.