1631130 - Record in telemetry when a login is recorded as used

Status: RESOLVED FIXED

(Toolkit :: Password Manager, task, P1)

Description

[Matthew N. [:MattN]]

We have big blind spot in password manager usage as we don't know when users are using a saved login other than via about:logins (copy/reveal) and autofill on page load (which doesn't mean the user actually submitted the form). We currently only know when a login is autofilled, saved or updated. Bug 1621520 will record filling via the context menu and autocomplete but that doesn't necessarily mean the saved login was submitted afterwards.

We do have PWMGR_LOGIN_LAST_USED_DAYS but it includes all logins put into buckets so can't easily tell you whether the user used any saved logins in a session.

Comment 1

[Matthew N. [:MattN]]

Ideally we would exclude manually typing the saved login values and know which UI/UIs was/were used to fill the login but we can probably add that to extra_keys later… for now we can include a boolean indicating whether any fill happened (using autoFilledLoginGuid) but it won't take any user edits after the fill into account.

It would also be nice to include whether the login was breached/vulnerable to align with bug 1622693.

Comment 2

[Matthew N. [:MattN]]

(In reply to Matthew N. [:MattN] (PM me if request are blocking you) from comment #1)

It would also be nice to include whether the login was breached/vulnerable to align with bug 1622693.

Both of these are kinda expensive to compute for every login used so I think I will leave this out for now.

Comment 3

[Matthew N. [:MattN]]

Attachment: Bug 1631130 - Record in telemetry when a login is recorded as used. r=sfoster

Also fix some incorrect/outdated histogram metadata.

Comment 4

[Matthew N. [:MattN]]

A patch is up but I am still working on tests.

Comment 5

[Ed Lee :Mardak]

We'll want this telemetry as part of bug 1617759 separate from bug 1628849 to measure when login forms are submitted to see if import suggestion results in more submissions.

Comment 6

[Matthew N. [:MattN]]

[Tracking Requested - why for this release]: See comment 5.

Comment 7

[Matthew N. [:MattN]]

Attachment: Request for data collection review form

It would be great if this could be reviewed ASAP so we can uplift this to 77. Sorry for the last minute request.

Comment 8

[Matthew N. [:MattN]]

(In reply to Matthew N. [:MattN] (PM me if request are blocking you) from comment #6)

[Tracking Requested - why for this release]: See comment 5.

Is this essential for 77? I've been busy and didn't realize the last beta was today. Not sure if this would be taken for RC, it does seem low risk.

Comment 9

[Ed Lee :Mardak]

Su says this is nice to have for the password import suggestion experiment and the main experiment goals are still fine with the telemetry that has already landed/uplifted to 77. So no need to rush this in.

Comment 10

[Chris H-C :chutten]

Comment on attachment 9150999 [details]
Request for data collection review form

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file [Events.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Events.yaml) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, :MattN is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c13f1b0f78ad
Record in telemetry when a login is recorded as used. r=sfoster

Comment 12

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/c13f1b0f78ad

Comment 13

[Timea Cernea [:tbabos][inactive]]

Hey Matt, we should see "pwmgr" "saved_login_used" when an autofilled login was submitted, right? Is this telemetry recorded once per session, per site? When you have the time, please fill in the important details to verify so we can check it out on Nightly.

Comment 14

[Matthew N. [:MattN]]

(In reply to Timea Cernea [:tbabos] from comment #13)

Hey Matt, we should see "pwmgr" "saved_login_used" when an autofilled login was submitted, right?

Yes, all uses where what was submitted matches a password (and sometimes username) that was saved for the site.

Is this telemetry recorded once per session, per site?

No, it's each use.

When you have the time, please fill in the important details to verify so we can check it out on Nightly.

There are a few follow-up dependencies already filed to take into account and I will write tests in bug 1640311. For now just do a basic check that any time a password is used via a form or an auth. dialog that the event is shown.