Closed Bug 715824 (b2g-a11y) Opened 13 years ago Closed 6 years ago

A11y in B2G

Categories

(Firefox OS Graveyard :: General, defect)

Product:
Firefox OS Graveyard
Component:
General
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(blocking-basecamp:-)

Status:
RESOLVED WONTFIX
Project Flags:
blocking-basecamp -

People

(Reporter: cjones, Assigned: eeejay)

References

Details

(Keywords: access, Whiteboard: [tech-p1]) 

The current plan as of my last chat with dbolter is to continue along the path of a "classic"-style extension being packaged in b2g chrome, reading the screen using our existing DOM interfaces, and synthesizing audio in JS.  Since the b2g homescreen will soon be rewritten in HTML, there are no more known obstacles.

We'll need to keep an eye on the quality and performance of the synthesizer.
Alias: b2g-a11y
A more optimized JS speech synthesizer can be seen at http://syntensity.com/static/espeak.html

This uses basically all the current emscripten optimizations. The next step for optimizing it is to wait for emscripten to support all LLVM optimizations (hopefully in a month or so). Although, we could probably make it faster just be running it a few times ahead of time (if you play text several times, it gets faster, I assume because TI analyzes the code better each time). Another speedup should happen when bug 706914 lands.

Note that the synthesizer uses a worker (to keep the main thread responsive), which causes a memory problem (bug 718100) at the moment.

Note also that for B2G, this would probably be much too slow without TI, which last I heard is disabled on mobile due to memory constraints. TI is really needed to make code like this fast.
Eitan, can you please link to your current wiki docs/plans?
Here are some a11y UX principles for mobile:
https://wiki.mozilla.org/Accessibility/Mobile/UserExperience

Here is what will be a mapping of common functionality to gestures, etc.
https://wiki.mozilla.org/Accessibility/Mobile/GestureInterface
Whiteboard: [b2g:blocking-]
No a11y requirements for V1, though it sounds like it may be a requirement for V2.

Leaving the blocking- flag for tracking purposes, and severing the dependency from the product phone tracker bug.
No longer blocks: b2g-product-phone
blocking-basecamp: --- → -
Whiteboard: [b2g:blocking-]
I still hold on to the dream of building our testing infrastructure around a11y.
Blocks: b2g-v-next
Assigning to Eitan to connect our dependent bugs.
Assignee: nobody → eitan
(In reply to Chris Jones [:cjones] [:warhammer] from comment #5)
> I still hold on to the dream of building our testing infrastructure around
> a11y.

afaik that should be possible today if you want to test through the nsIAccessible* interfaces.
Yep.  I'd really like us to get something WAI-ARIA wired up too.

(In reply to Chris Jones [:cjones] [:warhammer] from comment #5)
> I still hold on to the dream of building our testing infrastructure around
> a11y.

Also, oh golly, I hope it was clear, but implicit in that was, "in addition to a11y support itself"! :)  Building testing around a11y means it ain't gonna regress and has to be flawless.
(In reply to Chris Jones [:cjones] [:warhammer] from comment #8)
> Yep.  I'd really like us to get something WAI-ARIA wired up too.

I'm not sure what you mean by this, those interfaces expose information about the dom but aria and some layout stuff effects them too.
I'm way out of my depth here, so excuse my ignorance ;), but ideally I'd like to build something that uses standard-ish APIs so it can be used on other systems.  That's a nice-to-have, not a hard requirement, though.
Needed for competitive parity.
Whiteboard: [tech-p1]
(In reply to Chris Jones [:cjones] [:warhammer] from comment #10)
> I'm way out of my depth here, so excuse my ignorance ;), but ideally I'd
> like to build something that uses standard-ish APIs so it can be used on
> other systems.  That's a nice-to-have, not a hard requirement, though.


are you something like a legit web api instead of our custom gecko thing?  That would be interesting, and I've sort of idly tossed the idea around for a minute or two before, but I don't think anybodies really seriously proposed doing it before.  That said if we want to allow other people to write a11y apps for b2g its probably a good idea.

I believe webkit has its own custom interfaces that are fairly different from ours atleast in a 10 second glance.  It would be interesting to use such a web api to test that browser expose the same accessibility information for the same markup.
[CC+ Surkov]

I could ramble quite a lot about web a11y API thoughts (also, there are a few categories) - Chris would a live meeting/call help?

Although in the long term we may make use of more web facing 'a11y' API ourselves I think we need to consider our out-of-the-box working a11y solutions separately for v.next. AccessFu is going to be our awesome aural rendering of FF OS that we supply just as we supply visual rendering already (although AccessFu won't be activated by default, except maybe down the road if we partner with a car company or something).

(In reply to Trevor Saunders (:tbsaunde) from comment #12)
> (In reply to Chris Jones [:cjones] [:warhammer] from comment #10)
 
> I believe webkit has its own custom interfaces that are fairly different
> from ours atleast in a 10 second glance.  It would be interesting to use
> such a web api to test that browser expose the same accessibility
> information for the same markup.

I think Dominic would be amenable to meeting halfway if there was benefit. Heck maybe we could finally have an automated a11y acid test :)
It'd be awesome:
1) Running existing a11y mochitests on b2g devices
2) Firefox OS UI testing by triggering AccessFu methods and checking generated utterance

I guess conversion of nsIAccessible interfaces into web idl (adding a property to DOM window object for access to the a11y service) makes it a web api. It'd be cool to do that for Firefox OS to let people to invent.

Probably we should have a standard api one day but now it doesn't seem we have other benefits than running webkit a11y tests in firefox.
(In reply to David Bolter [:davidb] from comment #13)
> [CC+ Surkov]
> 
> I could ramble quite a lot about web a11y API thoughts (also, there are a
> few categories) - Chris would a live meeting/call help?
> 
> Although in the long term we may make use of more web facing 'a11y' API
> ourselves I think we need to consider our out-of-the-box working a11y
> solutions separately for v.next. AccessFu is going to be our awesome aural
> rendering of FF OS that we supply just as we supply visual rendering already
> (although AccessFu won't be activated by default, except maybe down the road
> if we partner with a car company or something).
> 

Is there any documentation about the referred AccessFu technology? how to try it? 

> (In reply to Trevor Saunders (:tbsaunde) from comment #12)
> > (In reply to Chris Jones [:cjones] [:warhammer] from comment #10)
>  
> > I believe webkit has its own custom interfaces that are fairly different
> > from ours atleast in a 10 second glance.  It would be interesting to use
> > such a web api to test that browser expose the same accessibility
> > information for the same markup.
> 
> I think Dominic would be amenable to meeting halfway if there was benefit.
> Heck maybe we could finally have an automated a11y acid test :)
Blocks: 525444
(In reply to Jose M. Cantera from comment #15)
> (In reply to David Bolter [:davidb] from comment #13)
> > [CC+ Surkov]
> > 
> > I could ramble quite a lot about web a11y API thoughts (also, there are a
> > few categories) - Chris would a live meeting/call help?
> > 
> > Although in the long term we may make use of more web facing 'a11y' API
> > ourselves I think we need to consider our out-of-the-box working a11y
> > solutions separately for v.next. AccessFu is going to be our awesome aural
> > rendering of FF OS that we supply just as we supply visual rendering already
> > (although AccessFu won't be activated by default, except maybe down the road
> > if we partner with a car company or something).
> > 
> 
> Is there any documentation about the referred AccessFu technology? how to
> try it? 
> 

If you go to developer settings on a Firefox OS phone with version 1.3 or later, you could turn on the screen reader and try it there.
Firefox OS is not being worked on
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.