961540 - Provide accessibility events when any accessibility service is active

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: General, enhancement)

Description

[Veeti Paananen]

I am building an app that makes use of an Android accessibility service to capture TYPE_VIEW_TEXT_CHANGED events. Changing the text in HTML input fields does not fire them in Firefox for Android. All native input fields, WebView and Chrome do this.

Comment 1

[Kevin Brosnan [Ex-Mozilla]]

Known issue?

Comment 2

[Eitan Isaacson [:eeejay]]

Veeti,

I think we intentionally suppress text changes that don't happen directly from user input. In other words, text changes done in js. Is that the case you are seeing? Does changing the text manually fire an event?

Comment 3

[Veeti Paananen]

OK, so I took a look at the source and it looks like there's some sort of whitelist for accessibility services in GeckoAccessibility. Because of this, no events are being sent.

Adding my service to the list makes input reporting work fine, but at the same time enables all sorts of other accessibility stuff that basically breaks the browser (I guess it's expecting a full screen reader to allow navigation, which I am not working on).

Perhaps it could be modified to always dispatch events, but keep the "rest of accessibility" to just the whitelist? I tried just naively removing the sEnabled check from sendAccessibilityEvent but that doesn't seem to be enough.

Comment 4

[Eitan Isaacson [:eeejay]]

(In reply to Veeti Paananen from comment #3)
> OK, so I took a look at the source and it looks like there's some sort of
> whitelist for accessibility services in GeckoAccessibility. Because of this,
> no events are being sent.
> 
> Adding my service to the list makes input reporting work fine, but at the
> same time enables all sorts of other accessibility stuff that basically
> breaks the browser (I guess it's expecting a full screen reader to allow
> navigation, which I am not working on).

Correct. For full context of why that whitelist is there see bug #779096. When Firefox detects an accessibility service it does more than simply emitting events. It goes into "screen reader mode". Since there are plenty of apps out there that use the accessibility API for simple automation tasks that are unrelated to actual accessibility, we created a whitelist of "legitimate" services.

For your app to work correctly we would need some intermediate mode that enables a11y events, but does not change the interaction model.

Comment 5

[Brad Lassey [:blassey] (use needinfo?)]

Eitan, I assume you're going to resolve this as invald. If not, please renom and explain

Comment 6

[Eitan Isaacson [:eeejay]]

When a non-whitelisted service is present, we should still be emitting accessibility events without activating our baked-in a11y-centric input mode. This will allow more general purpose accessibility services to get what they need. 

The NotificationListenerService introduced in API 18 was introduced so that notification enhancing apps don't exploit the a11y API for non-a11y uses. So the threat of non-a11y accessibility services will decline.

Comment 7

[Timvde]

This is apparently a problem for applications like Password Store[1]. I just wanted to put this back on the radar :)

[1] https://github.com/zeapo/Android-Password-Store/issues/176

Comment 8

[Marcos H. Alano]

This seems a major turn down for most (all?) password managers. I use Enpass and it's unable to autofill passwords. LastPass also doesn't autofill. May be this is a problem with accessibility apps itself (I don't use anyone, so I can't tell for sure) making this a huge problem for accessibility which is vital on an open source browser which should be very inclusive.

Comment 10

[Albert Scheiner [:alberts]]

... and Bitwarden:
https://github.com/bitwarden/mobile/issues/107

Comment 11

[Marcos H. Alano]

I think Autofill on Android Oreo can replace the accessibility feature quite well for lots of user cases except, of course, accessibility questions.

Comment 12

[Albert Scheiner [:alberts]]

Michael, any chance this might get some attention?

Comment 13

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Susheel, can you triage this?

Comment 14

[:sdaswani]

I'm going to punt to David as he can probably better coordinate this work.

Comment 15

[David Bolter [:davidb] (NeedInfo me for attention)]

Eitan is closest to the metal here. Eitan can you triage/update/resolve this bug?

Comment 16

[Eitan Isaacson [:eeejay]]

Password manager support is being worked on in bug 1330257. It will both support the legacy api as well as the oreo autofill api. This maybe can be closed as a DUP if the main concern here was a11y-consuming password managers.

Comment 17

[BMO Automation]

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.