1333344 - (fx-autofill-metrics) [meta] [Form Autofill] Implement metrics

Status: NEW

(Toolkit :: Form Autofill, task)

Description

[Luke Chang [:lchang] (inactive)]

This is the meta bug for all metrics bug of Form Autofill

Comment 1

[Steve Chung [:steveck]]

For the metrics, we could either simply leverage main ping[1], or implement our custom ping[2]. Since implementing custom ping would need several prerequisites and setup[2], it's seems not necessary for our MVP IMO especially the credit card autofill isn't started yet. But it might be good to know how much time/resource we'll need for custom ping anyway.

[1]http://gecko.readthedocs.io/en/latest/toolkit/components/telemetry/telemetry/collection/index.html
[2]http://gecko.readthedocs.io/en/latest/toolkit/components/telemetry/telemetry/collection/custom-pings.html#designing-custom-pings

Comment 2

[Steve Chung [:steveck]]

Hi Joe, before the data review meeting, we have some general questions that might need your confirmation:

- Do we need to query the information that need to combine multiple queries? For example, would you need to know the data about the form filling time when user decides to apply/not to apply the profile, something like that. It would be great if you can list out all the queries you want, and the results could help us to understand that whether users really benefit from the this feature.
- Do we need to have a configurable date or duration on the dashboard to view the result? It would be related to how we store the data.

Comment 3

[Steve Chung [:steveck]]

I could think of 3 different proposes for 3 different pings, and each ping will cover several user story:
- Collecting the form submitted time while feature enabled/disabled. It could help to understand whether users actually help them to reduce the filling time. I guess we'll need to be more clear about the "enabled" case(must have profile and submit the form without editing). It will cover bug 990201, bug 1341569.
- Collecting the number of users that enabled/disabled from pref or doorhanger. Having a enable rate might be straightforward to know if users like it. The number of disabled type might reveal that doorhanger is confusing if the doorhanger disabled rate is high, or users dislike the experience if pref disabled rate increased. It will cover bug 990199, bug 990200
- Collecting the number of profiles per user, which will be classified by the saved/used. It's for figuring out whether users created too many unused profiles due to the inaccurate heuristic. It will cover bug 990203, bug 990204

Comment 4

[Steve Chung [:steveck]]

(In reply to Steve Chung [:steveck] from comment #2)
> Hi Joe, before the data review meeting, we have some general questions that
> might need your confirmation:
> 
> - Do we need to query the information that need to combine multiple queries?
> For example, would you need to know the data about the form filling time
> when user decides to apply/not to apply the profile, something like that. It
> would be great if you can list out all the queries you want, and the results
> could help us to understand that whether users really benefit from the this
> feature.
> - Do we need to have a configurable date or duration on the dashboard to
> view the result? It would be related to how we store the data.

We'll add telemetry with Histograms.json and Scalars.yaml just like normal m-c way for now, since there no further requirement from PM and these pings should be sufficient for generating the report.

Comment 5

[Sylvestre Ledru [:Sylvestre]]

Inactive meta bug, closing.