1949920 - Windows Narrator does not always remember its place in paragraph

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[Nathan LaPré]

STR:

1. Turn on UIA support in about:config; set accessibility.uia.enable to true.
2. Launch Windows Narrator
3. Open site data:text/html,<p>This is a paragraph. Narrator should begin reading this paragraph when the user tabs to the button and then presses the keyboard shortcut Narrator + Down Arrow. Narrator should remember its place in this paragraph when the user presses the Narrator key, and restart from the remembered point when the user presses Narrator + Down again. Edge appears to make this work perfectly, but Firefox Nightly has issues with it in native UIA support.</p>
4. Press Narrator + l a few times to advance a few words in. For example, advance until Narrator speaks "paragraph" for the first time.
5. Press Narrator + Down Arrow to request that Narrator read the document.

Expected: Narrator begins reading from the point where you stopped it, as it does in Edge.
Actual: Narrator starts reading from the beginning of the paragraph.

Narrator correctly remembers its position if you navigate line-by-line or word-by-word, but this particular behavior - in what I imagine is a somewhat common use case - is broken. The workaround is simple enough, but avoiding losing your place seems important to me.

Comment 1

[James Teh [:Jamie]]

1. Implementing the UIA TextChild pattern doesn't seem to fix this.
2. I noticed that Narrator updates the caret (ITextRangeProvider::Select with a collapsed range) as the cursor moves in scan mode.
3. We do have a bug in our ITextRangeProvider::Select implementation which fails to update the caret if it is set outside of an editable control. I fixed that bug, but Narrator still doesn't move the caret; i.e. ITextRangeProvider::Select is never called.

Comment 2

[James Teh [:Jamie]]

When this fails, it looks like Narrator reports the name of the text leaf instead of using the text pattern. I determined this by fudging the UIA Name property to return a bogus string for text leaves and Narrator reports the bogus string when I do a say all.

Curiously, this works as expected if I change the <p> to an <ins>. The cursor tracks correctly and Narrator definitely uses the text pattern. The only difference in the UIA tree when I do this is that the <ins> returns true for IsControlElement, where <p> does not. But in Edge, <p> and <ins> both return false for isControlElement, so this really shouldn't be the reason.

In addition to comment 1, here are other things I've tried, all unfortunately without success:

1. Fired the AsyncContentLoaded event.
2. Exposed paragraphs as UIA Group instead of Text. Edge does this.
3. Stopped exposing the Text pattern on the root accessible. I was wondering whether it was walking up to the root for some reason and failing.
4. Exposed the URL as the Value on the document. Edge does this.
5. Fixed SupportedTextSelection on the document. Previously, it returned SupportedSelection_None, which is a real bug.
6. Implemented IScrollProvider. I didn't do a full implementation, but I returned success and 100.0 for all the things, which should have been enough to convince Narrator.
7. Made the parent of the document return false for IsControlElement.

Comment 3

[James Teh [:Jamie]]

Argh! I just discovered that text leaf nodes implement the Text pattern in Chromium, contrary to at least one part of the UIA documentation. This seems very suspicious given my previous findings and I suspect that may be the cause.

Comment 4

[James Teh [:Jamie]]

Yup, supporting the Text pattern on text leaves fixes this.

Comment 5

[James Teh [:Jamie]]

Attachment: Bug 1949920: Expose the UIA Text pattern on text leaves. Don't expose it on the root.

We need to support the Text pattern on text leaves to fix issues with Narrator.
Removing support on the root isn't strictly necessary, but the Text pattern isn't normally supported on a UIA Window and Chromium doesn't do this, so I think it's best to follow.

This change necessitated moving UiaText so that it is no longer inherited by ia2AccessibleHypertext.
I initially tried to inherit UiaText into uiaRawElmProvider, but that doesn't work because ITextProvider and ISelectionProvider both have a GetSelection method with the same signature.
This meant that the ISelectionProvider implementation of GetSelection in uiaRawElmProvider was overriding the implementation in UiaText.
Instead, UiaText has now been split into a separate object.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ab2b2d1eb6dc
Expose the UIA Text pattern on text leaves. Don't expose it on the root. r=nlapre

Comment 7

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/ab2b2d1eb6dc

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:Jamie, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox137 to wontfix.

For more information, please visit BugBot documentation.