1901459 - [UIA] Support embedded objects in text

Status: RESOLVED FIXED

(Core :: Disability Access APIs, task)

Description

[James Teh [:Jamie]]

We need to support two types of embedded objects for UIA text:

1. Embedded objects where the text is included by the text pattern. This allows UIA to retrieve semantic information, etc. Unlike IAccessible2, these are not represented by embedded object characters; the text is just flattened.
2. Embedded objects which can't be represented as text such as images. These are represented by embedded object characters.

In both cases, we need to support ITextProvider::RangeFromChild, ITextRangeProvider::GetChildren and ITextRangeProvider::GetEnclosingElement. For the second case only, we'll need to support exposing the embedded object character.

Comment 1

[James Teh [:Jamie]]

Attachment: Bug 1901459 part 1: Correctly remove any globals in the a11y Python environment after each test.

First, previously, we were doing this in registerCleanupFunction, but this actually runs after the entire test file has finished, not each task.
We need this to happen after each task because some tasks might set Python globals which might interfere with other tasks.

Second, even when this did run previously, we didn't actually wait until it was complete.
runPython() is async, so we need to wait for the Promise to be fulfilled.

Third, if we had awaited this previously, we would have hung forever.
The Python runner wasn't sending a response when it finished the reset, so the JS code never knew when it had completed.
We now just send a return of None in this case to signal completion.

Comment 2

[James Teh [:Jamie]]

Attachment: Bug 1901459 part 2: Implement ITextRangeProvider::GetEnclosingElement.

Comment 3

[James Teh [:Jamie]]

Attachment: Bug 1901459 part 3: Implement ITextRangeProvider::GetChildren.

Comment 4

[James Teh [:Jamie]]

Attachment: Bug 1901459 part 4: Implement ITextProvider::RangeFromChild.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/76b1d3d49c46
part 1: Correctly remove any globals in the a11y Python environment after each test. r=nlapre
https://hg.mozilla.org/integration/autoland/rev/619d77ab44af
part 2: Implement ITextRangeProvider::GetEnclosingElement. r=nlapre
https://hg.mozilla.org/integration/autoland/rev/9f33a4b8f721
part 3: Implement ITextRangeProvider::GetChildren. r=nlapre
https://hg.mozilla.org/integration/autoland/rev/fe39f6eef856
part 4: Implement ITextProvider::RangeFromChild. r=nlapre

Comment 6

[tszentpeteri]

https://hg.mozilla.org/mozilla-central/rev/76b1d3d49c46
https://hg.mozilla.org/mozilla-central/rev/619d77ab44af
https://hg.mozilla.org/mozilla-central/rev/9f33a4b8f721
https://hg.mozilla.org/mozilla-central/rev/fe39f6eef856

Comment 7

[James Teh [:Jamie]]

Attachment: Bug 1901459 fixup: Cosmetic changes from code review that I applied locally and somehow neglected to push to Phabricator.

Comment 8

[BugBot [:suhaib / :marco]]

A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/894939566451
fixup: Cosmetic changes from code review that I applied locally and somehow neglected to push to Phabricator. r=morgan

Comment 10

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/894939566451