1894849 - Semantic information in tagged PDF not exposed to screen readers

Status: VERIFIED FIXED

(Firefox :: PDF Viewer, defect, P1)

Description

[Timothy Wynn]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0

Steps to reproduce:

This bug is reproduceable in a new Firefox profile, though I do not know when it started. According to this article though, Firefox's PDF viewers is supposed to render the tags in a PDF to screen readers.

1. With NVDA (Windows screen reader), open a tagged PDF in Firefox, e.g.:
   https://www.ssa.gov/pubs/EN-05-10023.pdf
2. Press D to move to the first page, where the first piece of text should be "Social Security Numbers for Children".

Actual results:

This heading is broken into two elements, i.e.:

Social SecurityNumbers for

Children

Expected results:

NVDA should have announced that this was a level-one heading.

Comment 1

[James Teh [:Jamie]]

Confirmed. I originally wondered whether this might be a problem with the specific PDF, but: 1) Acrobat doesn't warn about an untagged PDF and it exposes the headings correctly; and 2) the simple test PDF doesn't expose headings correctly either. I'm pretty sure it used to; certainly, this is severe enough that it's highly unlikely that I missed this in the initial review of this work (bug 1705139).

I'm marking this as s3 for now, as you can still read the text, but it's bordering on an s2.

(In reply to Timothy Wynn from comment #0)

This heading is broken into two elements, i.e.:

Social SecurityNumbers for

Children

We were aware of this bug at least. It's bug 1708035. But I'm fairly sure we still exposed the fact that this was a heading previously, despite the broken block.

Comment 2

[James Teh [:Jamie]]

5:55.60 INFO: Last good revision: 18720cd9e180a84f2440b6cfd6bc283e5bc63613
5:55.61 INFO: First bad revision: cbafc7ee44a8630659515b0fa2b87e1dd5465168
5:55.61 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=18720cd9e180a84f2440b6cfd6bc283e5bc63613&tochange=cbafc7ee44a8630659515b0fa2b87e1dd5465168
This implicates bug 1886872. There were a few a11y related fixes in that pdf.js merge, but I'm not sure which one of them would have broken this specifically.

Comment 3

[James Teh [:Jamie]]

The problem is that .canvasWrapper now has aria-hidden="true", but .structTree is inside .canvasWrapper, which means it is also aria-hidden. aria-owns can't apply on aria-hidden subtrees, so all the structure information from tags is lost.

Elevating this to s2, since this means all semantic info is lost.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1886872

:calixte, since you are the author of the regressor, bug 1886872, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 5

[James Teh [:Jamie]]

This is really odd. The regression range suggests Firefox 126, but the reporter and I can both reproduce the problem in Firefox 125.0.3.

Comment 6

[James Teh [:Jamie]]

I can't reproduce it in 124 though, so it only started in 125. That still suggests I might have misidentified the regressing bug incorrectly... somehow.

[Tracking Requested - why for this release]: All accessibility semantics are lost when reading tagged PDF documents.

Comment 7

[James Teh [:Jamie]]

Aha, bug 1889118 merged this in beta.

Comment 8

[James Teh [:Jamie]]

It seems like we might need some a11y browser tests which do some spot checks using the a11y tree to catch regressions like this. I know pdf.js has tests, but they're not going to catch something like this because it relates to a11y tree rules.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The bug is marked as tracked for firefox126 (beta) and tracked for firefox127 (nightly). We have limited time to fix this, the soft freeze is in 6 days. However, the bug still isn't assigned.

:marco, could you please find an assignee for this tracked bug? Given that it is a regression and we know the cause, we could also simply backout the regressor. If you disagree with the tracking decision, please talk with the release managers.

For more information, please visit BugBot documentation.

Comment 10

[Marco Castelluccio [:marco]]

We will definitely fix this for 127. I don't know if we'll have enough time to fix it for 126 unfortunately, but we could uplift it for the 126 planned dot release.

Comment 11

[BMO Github Automation]

Attachment: [mozilla/pdf.js] Avoid to aria-hide the structure tree (bug 1894849) (#18044)

Comment 12

[Donal Meehan [:dmeehan]]

:calixte do you plan on cherry picking a fix for Fx126 that is safe to uplift?
do you also plan on landing the pdf update in time for 127 that goes to beta next week

Comment 13

[Marco Castelluccio [:marco]]

Calixte is off this week. The fix landed but was backed out because of a timeout (bug 1895324 comment 4).

Comment 14

[Calixte Denizet (:calixte)]

Attachment: Bug 1894849 - Make semantic info in tagged pdfs available for screen readers

Comment 15

[Phabricator Automation]

beta Uplift Approval Request

• User impact if declined: Users with screen readers won't be able to read tagged pdfs
• Code covered by automated testing: yes
• Fix verified in Nightly: no
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: See comment#0
• Risk associated with taking this patch: Low
• Explanation of risk level: Very small change
• String changes made/needed: No
• Is Android affected?: yes

Comment 16

[Calixte Denizet (:calixte)]

Attachment: Bug 1894849 - Make semantic info in tagged pdfs available for screen readers

Comment 17

[Phabricator Automation]

release Uplift Approval Request

• User impact if declined: Users with screen readers won't be able to read tagged pdfs
• Code covered by automated testing: yes
• Fix verified in Nightly: no
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: See comment#0
• Risk associated with taking this patch: Low
• Explanation of risk level: Very small change
• String changes made/needed: No
• Is Android affected?: yes

Comment 18

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/367b7e23d3cd

Comment 19

[Daniel Bodea [:danibodea]]

I can reproduce this issue in Release v125.0.3 and V126.0 and Beta v127.0b1; NVDA verbalizes:

• "page 1 region clickable, social security number for"
• "page 1 region clickable, children"

I can verify the fix in Nightly v128.0a1; NVDA verbalizes:

• "page 1 region, social security numbers for, heading level 1".
• "heading level 1, children"
  I have to mention that the last word of the title is read as a separate title, by itself.

VoiceOver in the affected build does not recognize the title as heading level 1 and reads each line of the title separately:

• "Social Security, clickable" and then "You are currently on a text element"
• "Numbers For, clickable" and then "You are currently on a text element"
• "Children, clickable" and then "You are currently on a text element".

VoiceOver in fixed build recognizes the title as heading level 1 and reads the whole title in one go while also announcing the link:

• "Heading level 1, 4 items, Social Security Numbers for Children htpps"//www.ssa.gov/" and then automatically says "You are currently on a heading level 1."

ORCA in the affected build: reads each row of the title separately as simple text
ORCA in the fixed build: reads each row of the title separately, but announces "heading level 1 each time".

It will be verified in Beta127 when it will be available.

Comment 20

[Daniel Bodea [:danibodea]]

Verification was also performed in Beta v127.0b2 in Windows10 + NVDA, MacOS11 + VoiceOver and Ubuntu22 + Orca. The same behavior is observed as described in the fixed builds paragraphs from the above comment.

Comment 21

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-release/rev/335c1153ef7e

Comment 22

[Daniel Bodea [:danibodea]]

I can confirm that the behavior in Release Candidate v126.0.1 is the same as the "fixed behavior" described in comment 19. The First page's title elements are properly recognized as Heading Level 1 in the case of Windows10 + NVDA, MacOS11 + VoiceOver and Ubuntu22 + Orca.

Comment 23

[Timothy Wynn]

I don't know the etiquette for this sort of thing, so please feel free to delete if I am just generating noise instead of contributing anything useful. I can confirm that on Windows, updating to 26.0.1 restores the semantic information and have tested with both the test document from the description as well as the far simpler simple test PDF linked to in Comment 1.