1758689 - Implement CTW hit testing

Status: RESOLVED FIXED

(Core :: Disability Access APIs, task)

Description

[James Teh [:Jamie]]

On Windows with CTW enabled, we can't currently hit test at all. This is because when CTW is enabled, clients make calls to RemoteAccessibles in the parent process, rather than making calls to the content process marshaled via COM.

On Linux and Mac, we use sync IPDL. This can still work with CTW enabled, but it still has the risk of deadlocks if there are any sync content -> parent IPDL calls.

We need to implement a solution for CTW which avoids deadlocks and long freezes if the content process is busy. Doing this with the cache is tricky due to z-index, etc.

Comment 1

[James Teh [:Jamie]]

Implementing hit testing properly using the cache is going to be a very difficult challenge. To enable us to ship sooner, an interim solution could be that we still do hit testing synchronously, but with a short timeout (100 ms?) to prevent deadlocks or long freezes. That has the least chance of regressions, but is very ugly and doesn't get us to our goal of avoiding all sync a11y calls to content. Even so, I suspect hit testing isn't used by things other than screen readers and magnifiers, so this is maybe acceptable for now.

Another possibility is to do something similar to Chromium. It dispatches an async hit test, but then falls back to walking the a11y subtree and checking if the coordinate is inside the bounds of each node. The fallback isn't accurate for complex layout involving z-index, overflow, etc. The next time a hit test query is received, if the rect from the last async hit test matches, it will return that node. The idea is that the first hit test might be inaccurate, but subsequent hit tests within the same node will be more accurate.

Aside from the initial inaccuracy, one potential concern with this second solution is retrieving the bounds for every Accessible. Because we walk the ancestors to calculate bounds, this tree walk would end up being O(n^2). I guess only profiling will tell us how much that matters on large pages.

Comment 2

[James Teh [:Jamie]]

In the CTW meeting today, we discussed another approach:

• Walk the RemoteAccessible cache, rather than relying on an async (or sync) method.
• Cache z-index.
• One way to deal with z-index is to always walk the entire tree before returning a result. This just means that every query will take as long as every other, regardless of where it is in the a11y tree.
• If we need to, we can do a naive cache siilar to tables. We could generate a more efficient data structure lazily when hit testing first happens; e.g. sort the tree by z-index. We then just throw it away whenever there is a change that might invalidate it, rather than trying to keep it up to date.
• We also need to figure out how to deal with overflow.

Comment 3

[Morgan Reschenberg [:morgan]]

In the most recent CtW meeting we discussed porting over the android hit testing approach, which relies on layout's GetFramesForArea method. This is what I'm starting with :)

Comment 4

[Morgan Reschenberg [:morgan]]

Attachment: Bug 1758689: Create and maintain a viewport cache for hittesting r?Jamie,emilio

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c2233b3830ef
Create and maintain a viewport cache for hittesting r=Jamie,emilio

Comment 6

[Cristina Cozmuta (:CrissCozmuta)]

Backed out for causing nsRefreshDriver related failures

Backout link
Push with failures
Link to failure log 1
Link to failure log 2
Link to failure log 3
Link to failure log 4
Link to failure log 5

Failure line :
Assertion failure: removed, at /builds/worker/checkouts/gecko/layout/base/nsRefreshDriver.cpp:1514
TEST-UNEXPECTED-FAIL | accessible/tests/mochitest/actions/test_anchors.html (finished) | application terminated with exit code 1

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9f1d0d70629c
Create and maintain a viewport cache for hittesting r=Jamie,emilio

Comment 8

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/9f1d0d70629c