875040 - Introduce screen reader desktop simulator extension

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia, defect)

Description

[Eitan Isaacson [:eeejay]]

This is useful for testing Gaia components for accessibility. There is already a pull request open. But obviously we do things in Bugzilla.

https://github.com/mozilla-b2g/gaia/pull/9466

I got a chance to fix it to work with hi-dpi laptop displays so we should be good.

Comment 1

[Eitan Isaacson [:eeejay]]

Attachment: Introduce screen reader addon

Comment 2

[Kevin Grandon :kgrandon]

Comment on attachment 752893 [details] [diff] [review]
Introduce screen reader addon

Hi Eitan,

I think this is really cool! My only concern is that I'm not 100% convinced that this should sit inside of the gaia codebase.

My personal recommendation would be to upload this to AMO or perhaps implement in the Firefox nightly tools. If we go the AMO route initially, we could always add a hook into the Gaia build system to download the plugin in the makefile.

I'll add feedback flags here for Vivien and Alex to get their thoughts on where this should live.

Comment 3

[Alexandre Poirot [:ochameau]]

Comment on attachment 752893 [details] [diff] [review]
Introduce screen reader addon

I share Kevin's feeling.

Your tool is awesome! I want gaia contributors to use it and help them discover it, but also, It doesn't make sense to host devtools in gaia.
I know we did added some kind of devtools in gaia, but they are meant to land in Firefox or in Simulator in near future.

What would be splendid, is to release this tool on AMO and ensure that it works against regular webpages. So that we end up improving the web and not only gaia contributor world.
(We can then add your addon to this list: https://addons.mozilla.org/fr/firefox/collections/mozilla/webdeveloper/)

Then, Eitan, what do you think about tuning the build system to easily install third party addons from the web; would that work for you?
Something like:
  * custom-tools.txt with xpi URLS to automagically download and install,
  * automatic install of any xpi available in tools/extensions/ folder,
  * any other smart idea in order to ease using 3rd party addons.

Comment 4

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

Comment on attachment 752893 [details] [diff] [review]
Introduce screen reader addon

I share Kevin and Alex opinion too. This is a great tool that should be available via the devtools panel.

But in order to educate me I wonder how it is supposed to work on a b2g devices right now. The pref is set to auto on the device but how does the user turn on screen reading capability? 

Do we need to add something somewhere? eeejay do you know how it works normally? Does screen reader software takes this output and does text to speech? If yes do they read some command line output or do they call the internal of the OS for it?

Jonas do you have any ideas / plans?

Comment 5

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

Jonas I would like to know if you have any plans for accessibility already ? Or if you know how it works / supposed to work at least?

Comment 6

[Eitan Isaacson [:eeejay]]

(In reply to Vivien Nicolas (:vingtetun) (:21) from comment #4)
> Comment on attachment 752893 [details] [diff] [review]
> Introduce screen reader addon
> 
> I share Kevin and Alex opinion too. This is a great tool that should be
> available via the devtools panel.
> 

That sounds fine. I'll spin this off as a separate project. And add things to Gaia to pull in external addons.

> But in order to educate me I wonder how it is supposed to work on a b2g
> devices right now. The pref is set to auto on the device but how does the
> user turn on screen reading capability? 
> 

The device now does not have text to speech. Furthermore, a lot of work needs to be put in to Gaia to make it work well with a screen reader. We have a built-in screen reader which is dubbed AccessFu, that could be turned on with a pref, but it won't speak much since there is no tts yet :) It we developed for Android, and the plan is to use it in b2g as well. I am the primary developer.

> Do we need to add something somewhere?

Eventually we need to add screen reader preferences into the prefs app. It was there early on, and we pulled it because it was virtually unusable. The litmus test for "usable", is being able to turn it off again via screen reader gestures once it is activated :)

That is what this addon is for, it is for getting Gaia and AccessFu in shape on the desktop. Once we have basic functionality we could add the preference back, and developers could test on devices as well. There is a parallel effort to get tts working on a b2g device, bug 863174 for gonk, and bug 525444 for the content API.

> eeejay do you know how it works
> normally? Does screen reader software takes this output and does text to
> speech? If yes do they read some command line output or do they call the
> internal of the OS for it?
> 

Hopefully I answered some of these questions above.

Comment 7

[Eitan Isaacson [:eeejay]]

(In reply to Alexandre Poirot (:ochameau) from comment #3)
> Comment on attachment 752893 [details] [diff] [review]
> Introduce screen reader addon
> 
> I share Kevin's feeling.
> 
> Your tool is awesome! I want gaia contributors to use it and help them
> discover it, but also, It doesn't make sense to host devtools in gaia.
> I know we did added some kind of devtools in gaia, but they are meant to
> land in Firefox or in Simulator in near future.
> 
> What would be splendid, is to release this tool on AMO and ensure that it
> works against regular webpages. So that we end up improving the web and not
> only gaia contributor world.
> (We can then add your addon to this list:
> https://addons.mozilla.org/fr/firefox/collections/mozilla/webdeveloper/)
> 
> Then, Eitan, what do you think about tuning the build system to easily
> install third party addons from the web; would that work for you?
> Something like:
>   * custom-tools.txt with xpi URLS to automagically download and install,
>   * automatic install of any xpi available in tools/extensions/ folder,
>   * any other smart idea in order to ease using 3rd party addons.

That all sounds great. I already uploaded it to AMO, and it is useful for testing any web app, not just Gaia. Of course, my motivation for having this in Gaia is to have it used by Gaia devs. But I like the idea of pulling in external addons.

https://addons.mozilla.org/en-US/firefox/addon/screen-reader-simulator/

Comment 8

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

I'll defer to the accessibility team here. I believe they already have plans that they are working on. Checking with David Bolter on if this is moving us in the right direction or if there is any other way that people can help out.

Comment 9

[David Bolter [:davidb] (NeedInfo me for attention)]

Yes Eitan owns our strategy here.

Comment 10

[Eitan Isaacson [:eeejay]]

I'm closing this bug, as bug 879563 introduces a method of pulling in an external addon, and it pulls in the screen reader one by default.