Telemetry probe for usage of about:logins UI interactions

RESOLVED FIXED in Firefox 42

Status

()

Firefox for Android
Logins, Passwords and Form Fill
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: ally, Assigned: ally)

Tracking

unspecified
Firefox 42
Points:
---

Firefox Tracking Flags

(firefox42 fixed)

Details

Attachments

(1 attachment)

Comment hidden (empty)
(Assignee)

Updated

3 years ago
Blocks: 1183225
(Assignee)

Updated

3 years ago
Assignee: nobody → ally

Comment 1

3 years ago
We already have UI telemetry for settings and menu items, so I don't actually know that we need a new probe for this... this probe would only capture the case where users directly type "about:logins" but I think that would be an edge case.
I think this came up after we've talked to capture how long users might stay on this page, i.e. to verify the performance issue? 

How can we track the event click "Menu->Tools->Logins"? I thought that would bring them to about:logins?

Either way, I feel we need to get more info around how users understand this tool, so can I ask for the sky :)?

- How many times users click on any of the login menu items, how many times do people "edit" a password, or "show the password" etc.
- Of course, also I'd like to know the amount of passwords per user (or average). If see see this increase over time, it will be an indicator that they feel comfortable with the feature and know how to use it. If we only see people going to the feature without doing anything or lets say only save 1-2 passwords per user, it probably means they don't see a value in the tool, or have issues using it.

Comment 3

3 years ago
(In reply to Barbara Bermes [:bbermes] from comment #2)
> I think this came up after we've talked to capture how long users might stay
> on this page, i.e. to verify the performance issue? 

Ah, I was just confused because there wasn't any context in this bug. I thought that it was just about a probe for "how often do users use this page?".

> How can we track the event click "Menu->Tools->Logins"? I thought that would
> bring them to about:logins?

Yes. This should be covered by our menu UI telemetry. You can see this in mfinkle's UI telemetry dashboard if you search for "logins":
https://www.dropbox.com/s/3izxcs8j2412chi/Screenshot%202015-07-16%2015.14.23.PNG?dl=0

> Either way, I feel we need to get more info around how users understand this
> tool, so can I ask for the sky :)?
>
> - How many times users click on any of the login menu items, how many times
> do people "edit" a password, or "show the password" etc.

Yes, we can add a new UI telemetry probe for this.

> - Of course, also I'd like to know the amount of passwords per user (or
> average). If see see this increase over time, it will be an indicator that
> they feel comfortable with the feature and know how to use it. If we only
> see people going to the feature without doing anything or lets say only save
> 1-2 passwords per user, it probably means they don't see a value in the
> tool, or have issues using it.

We do already have histogram data for this:
http://telemetry.mozilla.org/dist.html#measure=PWMGR_NUM_SAVED_PASSWORDS&max_channel_version=nightly%2F41&min_channel_version=nightly%2F38&product=Fennec&cumulative=0&use_submission_date=0&start_date=2015-05-11&end_date=2015-06-29&sanitize=1
(In reply to :Margaret Leibovic from comment #3)

> > - How many times users click on any of the login menu items, how many times
> > do people "edit" a password, or "show the password" etc.
> 
> Yes, we can add a new UI telemetry probe for this.
Great, thanks -- should I create a new bug for this?

> We do already have histogram data for this:
> http://telemetry.mozilla.org/dist.
> html#measure=PWMGR_NUM_SAVED_PASSWORDS&max_channel_version=nightly%2F41&min_c
> hannel_version=nightly%2F38&product=Fennec&cumulative=0&use_submission_date=0
> &start_date=2015-05-11&end_date=2015-06-29&sanitize=1

wonderful, this will help us find out if we can move the needle, i.e. define adaptability.

Comment 5

3 years ago
(In reply to Barbara Bermes [:bbermes] from comment #4)
> (In reply to :Margaret Leibovic from comment #3)
> 
> > > - How many times users click on any of the login menu items, how many times
> > > do people "edit" a password, or "show the password" etc.
> > 
> > Yes, we can add a new UI telemetry probe for this.
> Great, thanks -- should I create a new bug for this?

Since we already seem to have probes for the other items mentioned in this bug, let's just make this bug about probes for the UI of about:logins.

At the very least, let's get a UI telemetry probe for the actions that appear when you hit a login. We can follow existing patterns for context menu probes for this.

We could also get a probe to see how often users actually change their password when they go to the edit login page. It could also be interesting to know how often they hit the "show/hide" button.

Ally, feel free to chime in with other ideas, since you're the one who originally filed this bug.
Summary: Telemetry probe for usage of about:logins → Telemetry probe for usage of about:logins UI interactions
sounds good and I like your suggestions as well
(Assignee)

Comment 7

3 years ago
Since UI telemetry's dashboard has been down for a quite a while and will not be up for awhile and UITelemetry lacks the control of the expiration in version it is thought that a stand alone probe will do better for our usage here.  

Margaret, settings -> privacy -> Manage Logins drops you into aboutLogins.js/xhtml

Expiration in Firefox 55 should be enough data & then some since this is rolling out in 42.
(Assignee)

Comment 8

3 years ago
Created attachment 8636851 [details] [diff] [review]
isAboutLoginsUsedAtAll

Vlad, the goal of this probe is to ascertain if about:logins and its edit login page are actually used in the in wild, and perhaps what parts they might use most.

I don't know how you feel about instrumenting the context menu until we know that the page(s) itself are used and worth investment.

this probe tracks how the list view is shown, how often the edit view is shown, how often the logins are edited, and how often people use the show/hide button on edit logins page. I'm undecided if the last one is really worth measuring at this time.   

Thoughts ?
Attachment #8636851 - Flags: review?(vdjeric)
Comment on attachment 8636851 [details] [diff] [review]
isAboutLoginsUsedAtAll

We don't usually see UI elements being probed in such a way (using histograms). For UI events and sessions like this, we usually use some UI Telemetry.
Blocks: 1170786
No longer blocks: 1183225
Comment on attachment 8636851 [details] [diff] [review]
isAboutLoginsUsedAtAll

Review of attachment 8636851 [details] [diff] [review]:
-----------------------------------------------------------------

::: mobile/android/chrome/content/aboutLogins.js
@@ +198,5 @@
>        headerText.textContent = gStringBundle.GetStringFromName("editLogin.fallbackTitle");
>      }
>    },
>    _onSaveEditLogin: function() {
> +    Services.telemetry.getHistogramById("PWMGR_ABOUT_LOGINS_USAGE").add(LOGIN_EDITED);

do you want to add a sample every time a login is edited in the same session or limit to one-sample-per-session? it changes how the resulting histogram is interpreted

::: toolkit/components/telemetry/Histograms.json
@@ +8173,5 @@
>    },
> +  "PWMGR_ABOUT_LOGINS_USAGE": {
> +    "expires_in_version": "55",
> +    "kind": "enumerated",
> +    "n_values": 4,

you might want to give n_values room to grow. otherwise, you'll have to declare a new histogram if you need to add enum values in the future
Attachment #8636851 - Flags: review?(vdjeric) → review+
The resulting histogram will show you the relative popularity of each of the about:logins elements, and it will also provide absolute counts for how many times each was used (by the opt-in Telemetry population).

Is this what you wanted?
Flags: needinfo?(bbermes)

Comment 12

3 years ago
(In reply to Allison Naaktgeboren please NEEDINFO? :ally from comment #7)
> Since UI telemetry's dashboard has been down for a quite a while and will
> not be up for awhile and UITelemetry lacks the control of the expiration in
> version it is thought that a stand alone probe will do better for our usage
> here.  

For mobile UI telemetry, I usually just use this: https://people.mozilla.org/~mfinkle/uitelemetry/
(In reply to Vladan Djeric (:vladan) -- please needinfo! from comment #11)
> The resulting histogram will show you the relative popularity of each of the
> about:logins elements, and it will also provide absolute counts for how many
> times each was used (by the opt-in Telemetry population).
> 
> Is this what you wanted?

Great, yes. Thanks
Flags: needinfo?(bbermes)

Comment 14

3 years ago
What's left to do here? Can we just address the feedback from comment 10 and land this?
Flags: needinfo?(ally)
(Assignee)

Comment 15

3 years ago
(In reply to Vladan Djeric (:vladan) -- please needinfo! from comment #10)
> Comment on attachment 8636851 [details] [diff] [review]
> isAboutLoginsUsedAtAll
> 
> Review of attachment 8636851 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: mobile/android/chrome/content/aboutLogins.js
> @@ +198,5 @@
> >        headerText.textContent = gStringBundle.GetStringFromName("editLogin.fallbackTitle");
> >      }
> >    },
> >    _onSaveEditLogin: function() {
> > +    Services.telemetry.getHistogramById("PWMGR_ABOUT_LOGINS_USAGE").add(LOGIN_EDITED);
> 
> do you want to add a sample every time a login is edited in the same session
> or limit to one-sample-per-session? it changes how the resulting histogram
> is interpreted
> 
I believe the idea is to know how often users edit any passwords in each session. So I believe that would be the former.

Yes we can do that. Its currently tabled behind some more kidfox work since I can do that in teh window between aug 5 and 10th.
Flags: needinfo?(ally)
https://hg.mozilla.org/mozilla-central/rev/3e37f8f83299
Status: NEW → RESOLVED
Last Resolved: 3 years ago
status-firefox42: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 42
You need to log in before you can comment on or make changes to this bug.