Closed Bug 1650929 Opened 3 months ago Closed 2 months ago

[Doorhanger V2] Add telemetry event for username/password edits in the doorhanger

Categories

(Toolkit :: Password Manager, task, P2)

Desktop
All
task

Tracking

()

VERIFIED FIXED
mozilla80
Tracking Status
firefox79 --- verified
firefox80 --- verified

People

(Reporter: severin, Assigned: severin)

References

Details

Attachments

(2 files)

Add telemetry that reports how frequently PM doorhanger information is edited, to support our OKR of improving our recommended values.

Requirements

  • When the user selects 'save' or 'update', send an event with booleans 'did_edit_un' and 'did_edit_pw'
  • Typing or pasting into a field should set the did_edit_* boolean state to true
  • State should be reset for each doorhanger instance
  • No ID (or other information that could be correlated to a user) should be sent

Verification notes

  • Can be verified locally in about:telemetry (There are some edge cases that affect when about:telemetry gets updated. You probably know more about this than I do, but if not feel free to reach out.)
  • The event will be called doorhanger_submitted. See its definition for details
  • Ensure that the extra_keys did_edit_un and did_edit_pw match user behavior
  • Ensure that the sent object (save or update) matches the original state when the doorhanger was created (this will not change if the user modifies the doorhanger fields)
  • Ensure that exactly 1 event is sent every time the doorhanger is either saved or updated
See Also: 16414081641406
Blocks: 1650941
Severity: -- → N/A
Attached file Data review request
Flags: needinfo?(tdsmith)
Attachment #9163647 - Flags: data-review+
Attachment #9163647 - Flags: data-review+ → data-review?(tdsmith)
Comment on attachment 9163647 [details]
Data review request

lgtm; thanks!

--

1) Is there or will there be **documentation** that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in Events.yaml and the probe dictionary.

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the Firefox telemetry opt-out.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?

n/a; temporary collection

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, interaction data

5) Is the data collection request for default-on or default-off?

Default-on.

6) Does the instrumentation include the addition of **any *new* identifiers**?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

It's up to :severin to renew the collection as needed.

9) Does the data collection use a third-party collection tool?

No.
Flags: needinfo?(tdsmith)
Attachment #9163647 - Flags: data-review?(tdsmith) → data-review+

Comment on attachment 9161785 [details]
Bug 1650929 - add 'pwmgr doorhanger modified' telemetry; r=MattN

Beta/Release Uplift Approval Request

  • User impact if declined: No user impact (MattN addition: we won't be able to measure the impact of our doorhanger v2 improvements)
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): This is a telemetry only change. It is very low risk, and should be transparent to the user.

We have no alternative to uplifting this if we want to establish baseline numbers for one of our OKRs.

  • String changes made/needed: None
Attachment #9161785 - Flags: approval-mozilla-beta?
Attachment #9161785 - Attachment description: Bug 1650929 - add 'pmgr doorhanger modified' telemetry;r=MattN → Bug 1650929 - add 'pwmgr doorhanger modified' telemetry; r=MattN
Pushed by mozilla@noorenberghe.ca:
https://hg.mozilla.org/integration/autoland/rev/64eaf3f34725
add 'pwmgr doorhanger modified' telemetry; r=MattN
Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: mozilla79 → mozilla80

Have we verified that the incoming Telemetry data from Nightly looks sane so far?

Flags: needinfo?(severin.mozilla)

(In reply to Ryan VanderMeulen [:RyanVM] from comment #7)

Have we verified that the incoming Telemetry data from Nightly looks sane so far?

Yes: https://sql.telemetry.mozilla.org/queries/72863

Flags: needinfo?(severin.mozilla)

Comment on attachment 9161785 [details]
Bug 1650929 - add 'pwmgr doorhanger modified' telemetry; r=MattN

Thanks for the confirmation. Approved for 79.0 RC1.

Attachment #9161785 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

I updated the comment 0 with verification details. LMK if any of it could use clarification.

Flags: needinfo?(timea.babos)

Hey Severin,
I am getting "did_edit_un:true" for Save and Update doorhanger for when I select an option from the username autocomplete dropdown, the following 2 scenarios are not ok at this moment:

Open doorhanger -> select autocomplete value -> save == did_edit_un: false | Recording of the issue: https://streamable.com/hhblod
Open doorhanger -> modify username -> select autocomplete value -> save == did_edit_un: false | Recording of the issue: https://streamable.com/n8da95

Same goes for the Update scenarios.

Flags: needinfo?(timea.babos) → needinfo?(severin.mozilla)

That's my bad Timea, I conflated two bugs when I described those cases to you. These two cases won't need to be valid until bug 1650941 lands, which will update the telemetry to account for the new autocomplete functionality.

Flags: needinfo?(severin.mozilla)

No worries, I also missed to see Bug 1650941 to figure it out on my own.
Verified-fixed on latest Firefox Nightly 80 and Beta 79 on Windows 10, MacOS and Ubuntu 16.04 for the cases where we manually edit the username or password.

Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.