1666998 - Paragraph boundary doesn't work in Google Docs braille mode

Status: VERIFIED FIXED

(Core :: Disability Access APIs, defect, P2)

Description

[James Teh [:Jamie]]

Bug 1520779 added support for the paragraph boundary. This works well in normal textareas and contentEditables. Unfortunately, it doesn't work in Google Docs braille mode. Instead, retrieving the paragraph returns only a line.

While there are no line break characters in the text, inspecting the DOM reveals that Docs is using <br role="presentation"> to force a line break without exposing it to a11y. I guess they do this because their contentEditable is faked for a11y and it's very difficult to ensure that lines correspond exactly to what is visually rendered. (I'm pretty sure I saw bugs in the past where lines were out of sync, so that's probably why this change was made.)

STR (with NVDA):

1. Open this test case:
   data:text/html,<p style="white-space: pre-wrap;">a<span> <br role="presentation"></span>b
2. Press control+home to move the review cursor to the paragraph.
3. Press NVDA+control+z to open the NVDA Python console.
4. Enter this command and press enter:
   nav.IAccessibleTextObject.textAtOffset(0, 3)
   • Expected: (0, 3, 'a b')
   • Actual: (0, 2, 'a ')

The reason this happens is that layout stops on the <br> as it should, since it still exists in the DOM even though it's pruned for a11y.

This works as expected in Chrome. That suggests they don't use layout for paragraph boundaries. This is not unreasonable, since paragraphs don't really depend on visual information other than block vs inline.

I think we could do something similar. Rather than using FindOffset, we would walk the a11y tree back and forth, stopping at line break accessibles or display: block/inline-block, ensuring we don't walk out of the Accessible on which we were called. We would then call Transformoffset directly to convert any descendant offsets into offsets relative to the calling Accessible.

We might be able to make use of the Pivot class with our own custom match rule to walk the tree back and forth, since Pivot already knows how to walk the tree restricted to a specified root.

Comment 1

[Yura Zenevich [:yzen] (please use "needinfo")]

Adding P2 priority, based on what bug 1520779 was, feel free to change.

Comment 2

[Marco Zehe (:MarcoZ)]

Attachment: Bug 1666998 - Implement layout-independent paragraph offset search, r?jamie!

This uses the Pivot class to search backwards and forwards from the current offsets until either end is reached, or an accessible that is either a line break or has a block frame, is found, and therefore constitutes a boundary of interest.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1b8515b4548c
Implement layout-independent paragraph offset search, r=Jamie

Comment 4

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2fbd8c58de2a
Backed out changeset 1b8515b4548c for bustages on HyperTextAccessible.cpp. CLOSED TREE

Comment 5

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Csuccess%2Cretry%2Crunning%2Cpending%2Crunnable&searchStr=android%2Cbuild&revision=1b8515b4548c5e9c4eb6b8d668755e883a66f929&selectedTaskRun=Jv9-LloKSkq_XQae0xh_JA.0

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=317265501&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/2fbd8c58de2a

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2a72b40ba26a
Implement layout-independent paragraph offset search, r=Jamie

Comment 7

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/2a72b40ba26a

Comment 8

[James Teh [:Jamie]]

Verified fixed (using Google Docs) in Firefox 83 Nightly build 20201004212809 on Windows with NVDA's "Read by paragraph" braille setting enabled.

Comment 9

[Camelia Badau [:cbadau], Desktop Test Engineering]

Based on comment 8, I will set firefox83 flag as verified.