670087 - AccessibleObjectFromPoint returns incorrect accessible for popup menus

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[James Teh [:Jamie]]

Str:
1. Press alt+f to activate the File menu.
2. Get the location of the "New Tab" menu item.
3. Call AccessibleObjectFromPoint() with a point covered by the location retrieved in step 2.
Expected: The accessible for the "New tab" menu item should be returned.
Actual: Some other accessible outside the menu is returned. (I'm guessing this accessible is overlapped by the menu item.)

The popup window containing the menu (window class name: MozillaDropShadowWindowClass) returns the Firefox top level frame when queried for its client (OBJID_CLIENT). IAccessible::accHitTest on the Firefox top level frame doesn't seem to know about popup menu items.

The MozillaDropShadowWindowClass window should return the popup menu accessible as its client. This menu accessible should answer IAccessible::accHitTest correctly for the menu and its items.

If that isn't an option, you can probably get away with having IAccessible::accHitTest on the top level frame answer for the menu and its items.

Comment 1

[James Teh [:Jamie]]

Impact: This breaks mouse tracking for Firefox menu items with NVDA.
Related NVDA ticket: http://www.nvda-project.org/ticket/1647

Comment 2

[alexander :surkov (:asurkov)]

the same issue as in bug 480347. I think it's reasonable to follow Jamie's suggestion in any case so keeping open.

Comment 3

[alexander :surkov (:asurkov)]

Robert, what's the best way to find popup content from window (nsWindow::ProcessMessage) created for popup?

Comment 4

[Robert O'Callahan (:roc) (email my personal email if necessary)]

get the window's view's frame.

Comment 5

[Michael Herrmann]

Hi, are there any news on this bug? We are developing a GUI automation tool and are having trouble with menus, particular in Thunderbird.

Comment 6

[alexander :surkov (:asurkov)]

nominating as q2 target

Comment 7

[alexander :surkov (:asurkov)]

Robert, one more question please. We use nsLayoutUtils::GetFrameForPoint on document root frame (http://mxr.mozilla.org/mozilla-central/source/accessible/src/generic/Accessible.cpp#838) but it seems doesn't take into account any popups shown above the document. How can I detect that the given point points to an element inside popup (originally the point is in screen coordinates)?

Comment 8

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Use nsLayoutUtils::GetPopupFrameForEventCoordinates(rootPresContext, aEvent). You can look at how PresShell::HandleEvent does this.

Comment 9

[alexander :surkov (:asurkov)]

Attachment: patch

Comment 10

[Trevor Saunders (:tbsaunde)]

Comment on attachment 761495 [details] [diff] [review]
patch

>     void* uniqueID = reinterpret_cast<void*>(-aVarChild.lVal);
> 
>     // Document.
>     if (IsDoc())
>-      return AsDoc()->GetAccessibleByUniqueIDInSubtree(uniqueID);
>+      return Document()->GetAccessibleByUniqueIDInSubtree(uniqueID);

what is reason for change?

>+#define NS_LOG_WMGETOBJECT(aWnd, aHwnd, aAcc)                                  \
>+  PR_LOG(gWindowsLog, PR_LOG_ALWAYS,                                           \
>+         ("Get the window:\n  {\n     HWND: %d, parent HWND: %d, wndobj: %p,\n",\
>+           aHwnd, ::GetParent(aHwnd), aWnd));                                  \
>+  PR_LOG(gWindowsLog, PR_LOG_ALWAYS, ("     acc: %p", aAcc));                  \
>+  if (aAcc) {                                                                  \
>     nsAutoString name;                                                         \
>-    acc->GetName(name);                                                        \
>+    aAcc->GetName(name);                                                       \

use Name()?

>-nsWindow::GetRootAccessible()
>+nsWindow::GetAccessible()
> {
>+  nsView* view = nsView::GetViewFor(this);
>+  if (view) {
>+    nsIFrame* frame = view->GetFrame();
>+    if (frame && nsLayoutUtils::IsPopup(frame)) {
>+      nsCOMPtr<nsIAccessibilityService> accService =
>+        services::GetAccessibilityService();
>+      if (accService) {
>+        a11y::DocAccessible* docAcc =
>+          GetAccService()->GetDocAccessible(frame->PresContext()->PresShell());
>+        if (docAcc) {
>+          NS_LOG_WMGETOBJECT(this, mWnd,
>+                             docAcc->GetAccessible(frame->GetContent()));
>+          return docAcc->GetAccessible(frame->GetContent());

roc I'm assuming this stuff seems reasonable to you.  I'd worry a little this might slow down the no a11y case some, but but what becomes GetRootAccessible() has already been pretty slow and could be easily speed up if someone cared I suspect so I'm not really concerned.

Comment 11

[alexander :surkov (:asurkov)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b01a988c64b7

Comment 12

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/b01a988c64b7