1826264 - [CtW] Support IAccessibleText::scrollSubStringTo

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[James Teh [:Jamie]]

From Vispero (JAWS vendor):

We discovered a regression in Nightly related to a scrolling using the IAccessibleText interface. To provide visual tracking of the current location JAWS tells the browser to scroll the virtual cursor location into view and draws a red rectangle around the text being currently spoken. This works nice in Firefox, but it is currently broken in Nightly. JAWS calls IAccessibleText::scrollSubstringTo on the desired element, and the call returns E_NOTIMPL in Nightly.

Comment 1

[James Teh [:Jamie]]

I don't think we need to block shipping of CtW on this, but I'd like to get it uplifted into 113 if I can.

Comment 2

[James Teh [:Jamie]]

Eitan, i see that Mac uses ScrollSubstringTo as well. However, it seems to always treat the range as HyperText offsets, even though CAchedTextMarkers use leaf offsets, not HyperText offsets. Is this a bug or is there a conversion somewhere I'm missing?

To support ScrollSubstringTo for CtW, we could simply support the ScrollSubstringTo IPDL method on Windows just as we do on Linux and Mac. However, we're going to need to support it for TextLeafRange at some point (if not already for Mac) and i don't think there's anything blocking that, so it probably makes to just go the whole way here.

To do that, we'll need to:

1. Add TextLeafPoint::ToDOMPoint, based on HyperTextAccessible::OffsetToDOMPoint.
2. Add TextLeafRange::ToDOMRange, based on TextRange::AssignDOMRange. This will involve moving (or temporarily duplicating) a bunch of utility methods as well.
3. Add TextLeafRange::ScrollIntoView, based on TextRange::ScrollIntoView.
4. Teach TextLeafRange how to construct from serialised TextRangeData.
5. Add PDocAccessible::ScrollTextLeafRangeIntoView. Make TextLeafRange::ScrollIntoView use that for RemoteAccessible.
6. Add HyperTextAccessibleBase::ScrollSubstringTo, calling TextLeafRange::ScrollIntoView.
7. Tweak the various platforms to use HyperTextAccessibleBase or TextLeafRange.

Comment 3

[Eitan Isaacson [:eeejay]]

In mac this works because the gecko accessible in mozTextAccessible is a hypertext and the offsets given are relative to it (with simple textareas and flat contenteditables).

Comment 4

[Eitan Isaacson [:eeejay]]

I'll take a look at this.

Comment 5

[Eitan Isaacson [:eeejay]]

Attachment: Bug 1826264 - P1: Implement TextLeafPoint::ToDOMPoint. r?Jamie

Generalized code we already had for getting DOM points for DOM
selection.

Comment 6

[Eitan Isaacson [:eeejay]]

Attachment: Bug 1826264 - P2: Add range TextLeafRange::ScrollIntoView. r?Jamie

Use it when caching is enabled.

Depends on D174976

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e17169847f88
P1: Implement TextLeafPoint::ToDOMPoint. r=Jamie
https://hg.mozilla.org/integration/autoland/rev/2d1fdd7362d9
P2: Add range TextLeafRange::ScrollIntoView. r=Jamie

Comment 8

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/e17169847f88
https://hg.mozilla.org/mozilla-central/rev/2d1fdd7362d9

Comment 9

[James Teh [:Jamie]]

(In reply to Eitan Isaacson [:eeejay] from comment #3)

In mac this works because the gecko accessible in mozTextAccessible is a hypertext and the offsets given are relative to it (with simple textareas and flat contenteditables).

What if we're dealing with a compound contenteditable? Do we need an additional fix here for Mac to deal with those?

Comment 10

[James Teh [:Jamie]]

Comment on attachment 9327580 [details]
Bug 1826264 - P1: Implement TextLeafPoint::ToDOMPoint. r?Jamie

Beta/Release Uplift Approval Request

• User impact if declined: Broken visual scrolling with JAWS screen reader.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Only impacts scrolling of text for a11y. Covered by both new and existing automated tests.
• String changes made/needed:
• Is Android affected?: No

Comment 11

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9327580 [details]
Bug 1826264 - P1: Implement TextLeafPoint::ToDOMPoint. r?Jamie

Approved for 113.0b5.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/885a803053ef
https://hg.mozilla.org/releases/mozilla-beta/rev/e27961a87df5