1232745 - Learn what parts of accessibility API are needed for different tools (telemetry)

Status: NEW

(Core :: Disability Access APIs, defect)

Description

[David Bolter [:davidb] (NeedInfo me for attention)]

Accessibility instantiation and the tax of running our accessibility code is currently an all or nothing deal. As part of figuring out what to do about this we should collect info on how our API is used and by which tool.

(Possibly useful for things like Bug 1163004)

Comment 1

[David Bolter [:davidb] (NeedInfo me for attention)]

Benjamin, what is the right way to go about this? I want API usage data correlated with a11y tool detection. We know how to identify some clients on Windows (by sniffing what dlls are injected in our process), and they may be other config that helps. So to start I'm thinking of instrumenting (possibly the entire) a11y API with telemetry. I'll want this to be per session I think...

Comment 2

[Jim Mathies [:jimm]]

For the win10 work api use logging to printf_stderr should suffice for local debugging.

Comment 3

[David Bolter [:davidb] (NeedInfo me for attention)]

(In reply to Jim Mathies [:jimm] from comment #2)
> For the win10 work api use logging to printf_stderr should suffice for local
> debugging.

Agreed. I think Bob still wanted API usage date in order to prioritize the caching work but I'm not sure actually. Bob?

Comment 4

[Bob Owen (:bobowen)]

To my mind it will be useful to know what is triggering accessibility (may not always be possible I guess).
I doubt that we would ever tailor behaviour to individual tools, but there might be different categories of things where we could behave differently ... OSK versus screen reader for example.

It would also be very useful to know what information they call for and the timing of this.
By timing I mean from after the first accessibility call or from after whatever event we sent (e.g. page load).

That way we can have some idea of what sort of "service level agreement" the tools are expecting.

I suppose just having this logged locally would help to get us started, but I think real world telemetry will be needed to understand what's really going on eventually.
Particularly whether we're actually meeting the needs of the tools in real browsing situations.

I think Jim might be talking about logging for investigating a specific issue with Windows 10 triggering accessibility in lots of cases.

At this point I don't really have a grasp of how long it would take to put even the simplest logging/telemetry in place.

It was interesting in that email from google that they were saying that they are lazily caching things, which is not what I understood.
This would lead me to think that the initial call for information is not what is killing performance, but the constant request for updates. Although it sounds like they are still working through performance issues.
Which leads us back to the need for some sort of telemetry, to know what solutions are going to work best.

Comment 5

[Jim Mathies [:jimm]]

(In reply to Bob Owen (:bobowen) from comment #4)
> I think Jim might be talking about logging for investigating a specific
> issue with Windows 10 triggering accessibility in lots of cases.

Yep, I'm particularly interested in trying to understand why touch screen devices load a11y and after they do, what they do with it.

Comment 6

[Benjamin Smedberg]

I don't think I can answer any questions about telemetry until you have the problem statement nailed down and figured out what data you want to collect. From the discussion so far it sounds like we'd be ok with just hand-collected logs from a small selection of users.