1744315 - Support explore by touch in fission frames

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect, P2)

Description

[Eitan Isaacson [:eeejay]]

Our current explore by touch code sends an IPC message to the document of the currently focused accessible (or top level doc). If the doc has OOP iframes in it, it won't be able to drill down further because it doesn't have access to that tree.

The long term solution is to have hittesting happen in the parent process. Since that won't be ready in a while, a medium term solution is to dispatch special mouse events that will use widget/apz paths to get to the right content doc. Then listen for those events in content docs and perform explore by touch.

Comment 1

[Eitan Isaacson [:eeejay]]

Attachment: Bug 1744315 - P1: Introduce chrome-only MozMouseExploreByTouch event. r?botond

The goal of this event is for doing hit testing in content and emit an
accessibility event with the result for Android's explore by touch.

This event allows us to use APZ's ability to target the correct content
doc.

Comment 2

[Eitan Isaacson [:eeejay]]

Attachment: Bug 1744315 - P2: Don't traverse into remote accessibles in explore by touch. r?Jamie

Our default traversal rule will eventually work with remote trees, but
explore by touch will be local-only until we do hittesting in the parent
process. This change prevents the parent process from drilling down past
outer docs into frames.

Depends on D132840

Comment 3

[Eitan Isaacson [:eeejay]]

Attachment: Bug 1744315 - P3: Do explore by touch via DOM event. r?Jamie,botond

Depends on D132841

Comment 4

[Botond Ballo [:botond]]

(In reply to Eitan Isaacson [:eeejay] from comment #0)

The long term solution is to have hittesting happen in the parent process. Since that won't be ready in a while, a medium term solution is to dispatch special mouse events that will use widget/apz paths to get to the right content doc. Then listen for those events in content docs and perform explore by touch.

Out of curiosity:

• What is the downside of the medium-term approach?
• Do you have an idea of how the parent process hit test will work?

Comment 5

[Eitan Isaacson [:eeejay]]

(In reply to Botond Ballo [:botond] from comment #4)

(In reply to Eitan Isaacson [:eeejay] from comment #0)

The long term solution is to have hittesting happen in the parent process. Since that won't be ready in a while, a medium term solution is to dispatch special mouse events that will use widget/apz paths to get to the right content doc. Then listen for those events in content docs and perform explore by touch.

Out of curiosity:

• What is the downside of the medium-term approach?

No immediate downside, we will need parent side hit testing for other platforms because they are sync APIs. So at that point we should use a single implementation to simplify things.

• Do you have an idea of how the parent process hit test will work?

Yeah, Morgan is working on caching accessible bounds in the parent process. So we will be reinventing the wheel and implement hit testing on top of that.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5b8b5608f4a8
P1: Introduce chrome-only MozMouseExploreByTouch event. r=botond
https://hg.mozilla.org/integration/autoland/rev/51a98c438c34
P2: Don't traverse into remote accessibles in explore by touch. r=Jamie
https://hg.mozilla.org/integration/autoland/rev/dc22e26fdf2c
P3: Do explore by touch via DOM event. r=Jamie,botond

Comment 7

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/5b8b5608f4a8
https://hg.mozilla.org/mozilla-central/rev/51a98c438c34
https://hg.mozilla.org/mozilla-central/rev/dc22e26fdf2c

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:eeejay, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.