Open Bug 1343317 Opened 3 years ago Updated 4 months ago

Investigate making GeckoView accessible on UiAutomator (DOM Tree visible)

Categories

(GeckoView :: General, defect, P2)

Unspecified
Android

Tracking

(Not tracked)

People

(Reporter: njpark, Assigned: eeejay)

References

(Blocks 1 open bug)

Details

With a lot of backlog on Fennec for Android side ( https://trello.com/b/a47BZyic/android-browser-core), now would be a good time to revisit the automation testing approach.

Google’s UiAutomatorviewer app shows the DOM tree for the Fennec, and for the id=layer_view, it has 3 GeckoView subviews. If we can make GeckoView display its sub-views, then Appium/UiAutomator/Espresso test framework would be able to see and interact with those elements as well as the chrome view.  My understanding is that since GeckoView is a kind of custom view, if we can implement the required methods, it should be able to expose its subviews.  

The necessary tasks would include: 
Handle directional controller clicks
Implement accessibility API methods
Send AccessibilityEvent objects specific to your custom view
Populate AccessibilityEvent and AccessibilityNodeInfo for your view
For more detail, see https://developer.android.com/guide/topics/ui/accessibility/custom-views.html

For this Bug, all we want to know is whether it is actually possible, and how much time would be needed to do it. A Fennec developer familiar with GeckoView implementation would be able to assess accurately.
Summary: Investigate making GeckoView accessible → Investigate making GeckoView accessible on UiAutomator (DOM Tree visible)
If this is possible, then we can use Google or Amazon's device farm to run tests on the nightly builds.
Now that Focus/Klar is switching from Chromium WebView to GeckoView, their UI tests are blocked on this bug.
Whiteboard: [geckoview:klar]
P3 because James says this will be a lot of work. We need to figure out what Klar's UI tests are specifically trying to do.
Priority: -- → P3
Eitan I'm guessing you can own this for next steps.
Assignee: nobody → eitan
Yup. I'll look into this.
UIAutomator does use Android's accessibility API.

https://android.googlesource.com/platform/frameworks/uiautomator/+/master/src/com/android/uiautomator/core/UiDevice.java#760

For this to work, we will indeed need a proper accessibility hierarchy in geckoview. We are currently debating our a11y support approach. The fact that we will want uiautomator support puts more emphasis on a full hierarchy implementation.
Accessibility support in GeckoView is a blocker and considered P1. We need to morph this bug or create a new bug that is more clearly understandable as Accessibility support blocks GV ship.
Flags: needinfo?(eitan)
Created an a11y api metabug.
Blocks: 1448921
Flags: needinfo?(eitan)
Product: Firefox for Android → GeckoView
OS: Unspecified → Android
Whiteboard: [geckoview:klar]
Rank: 20
You need to log in before you can comment on or make changes to this bug.