Uplift prerelease password manager telemetry probes into release (in service of Lockbox)

RESOLVED FIXED in Firefox 60

Status

()

RESOLVED FIXED
11 months ago
11 months ago

People

(Reporter: loines, Assigned: loines)

Tracking

unspecified
mozilla61
Points:
---

Firefox Tracking Flags

(firefox60 fixed, firefox61 fixed)

Details

Attachments

(3 attachments)

(Assignee)

Description

11 months ago
Posted file lockbox-request.md
I'd like to get some existing prerelease telemetry probes for the password manager uplifted into Release, see attachment (data review request) for details. 

This is for Lockbox - we are launching first as an iOS app that syncs the contents of the desktop password manager one-way to the app. Because the app will be read-only to start, any credential management will have be done through the desktop browser. We'd like to know (for example) whether Lockbox users are engaging more with pw manager after getting their credentials into the mobile Lockbox app. 

chutten has graciously helped me to get started on this.
(Assignee)

Updated

11 months ago
Flags: needinfo?(chutten)

Comment 1

11 months ago
First comment, as a Telemetry Module Owner:

The PWMGR_* probes are a little crusty with age. They'll need some sprucing up to be useful and productive members of data collection. So in addition to adding "releaseChannelCollection": "opt-out" to the definitions in Histograms.json for which you're requesting release channel collection, you will need to

* Add a "bug_numbers" field with at least this bug number in it (remove the probe from the bug_numbers whitelist in histogram-whitelists.json)
* Add an "alert_emails" field with at least :loines' email address in it (remove the probe from the alert_emails whitelist in histogram-whitelists.json)
* Check whether you actually want these probes to be recorded in both the "main" and "content" processes (I'm guessing these probes are only in the "main" process?) and editing the "record_in_processes" field appropriately.

(For instance, according to bug 1423446, the following PWMGR_* probes have never accumulated in the content process:
PWMGR_BLOCKLIST_NUM_SITES
PWMGR_LOGIN_LAST_USED_DAYS
PWMGR_MANAGE_OPENED
PWMGR_MANAGE_VISIBILITY_TOGGLED
PWMGR_NUM_HTTPAUTH_PASSWORDS
PWMGR_NUM_PASSWORDS_PER_HOSTNAME
PWMGR_NUM_SAVED_PASSWORDS
PWMGR_PROMPT_REMEMBER_ACTION
PWMGR_PROMPT_UPDATE_ACTION
PWMGR_SAVING_ENABLED
PWMGR_USERNAME_PRESENT)

Feel free to use me as a reviewer for the Firefox patch to polish these probes up :)

Comment 2

11 months ago
(Now, with my Data Steward hat on)

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, standard Telemetry mechanisms.

    Is there a control mechanism that allows the user to turn the data collection on and off? 

Yes, standard Telemetry mechanisms.

    If the request is for permanent data collection, is there someone who will monitor the data over time?**

Yes, :loines.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction Data.

    Is the data collection request for default-on or default-off?

Default-on.

    Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

    Is the data collection covered by the existing Firefox privacy notice? 

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No.

---
Result: datareview+
Flags: needinfo?(chutten)
Comment hidden (mozreview-request)
(Assignee)

Comment 5

11 months ago
Hope I did what I supposed to there... FYI I'm PTO tomorrow so if there's changes I'll get to them Monday - thanks Chris!
(Assignee)

Updated

11 months ago
Assignee: nobody → loines
Status: NEW → ASSIGNED

Comment 6

11 months ago
mozreview-review
Comment on attachment 8969444 [details]
Bug 1454733 - Add opt-out flag, alert email and bug numbers to PWMGR probes;

https://reviewboard.mozilla.org/r/238202/#review244304
Attachment #8969444 - Flags: review?(chutten) → review+

Comment 7

11 months ago
mozreview-review
Comment on attachment 8969445 [details]
Bug 1454733 - Remove PWMGR probes from bug number and alert email whitelists;

https://reviewboard.mozilla.org/r/238200/#review244306
Attachment #8969445 - Flags: review?(chutten) → review+
(Assignee)

Updated

11 months ago
Keywords: checkin-needed

Comment 8

11 months ago
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/762057fcd6dd
Add opt-out flag, alert email and bug numbers to PWMGR probes; r=chutten
https://hg.mozilla.org/integration/autoland/rev/25a6ce6ffd06
Remove PWMGR probes from bug number and alert email whitelists; r=chutten
Keywords: checkin-needed

Comment 9

11 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/762057fcd6dd
https://hg.mozilla.org/mozilla-central/rev/25a6ce6ffd06
Status: ASSIGNED → RESOLVED
Last Resolved: 11 months ago
status-firefox61: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla61
(Assignee)

Comment 10

11 months ago
Comment on attachment 8969445 [details]
Bug 1454733 - Remove PWMGR probes from bug number and alert email whitelists;

Approval Request Comment
[Feature/Bug causing the regression]: NA
[User impact if declined]: Lockbox will have to wait until near its Launch date (June, same time as 61) to collect data on pw manager usage in the browser, which might slow the R&D of new features that could be added to the pw manager in h2 2018. Basically, we'd like to collect baseline PWMGR data before we launch so we know what current usage is like. 
[Is this code covered by automated tests?]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3548132a5f566c14dd00b4aafcb3f45a844b1b51
[Has the fix been verified in Nightly?]: Can't be completely verified since the change only affects release versions.
[Needs manual test from QE? If yes, steps to reproduce]: No
[List of other uplifts needed for the feature/fix]: https://reviewboard.mozilla.org/r/238202/ (also in this bug)
[Is the change risky?]: I don't believe so
[Why is the change risky/not risky?]: It only contains changes to JSON fields that control whether the telemetry probes are recorded in release versions, as well as flags that link to this bug and add my email as a contact
[String changes made/needed]: None
Attachment #8969445 - Flags: approval-mozilla-beta?
(Assignee)

Comment 11

11 months ago
Comment on attachment 8969444 [details]
Bug 1454733 - Add opt-out flag, alert email and bug numbers to PWMGR probes;

Approval Request Comment
[Feature/Bug causing the regression]: NA
[User impact if declined]: Lockbox will have to wait until near its Launch date (June, same time as 61) to collect data on pw manager usage in the browser, which might slow the R&D of new features that could be added to the pw manager in h2 2018. Basically, we'd like to collect baseline PWMGR data before we launch so we know what current usage is like. 
[Is this code covered by automated tests?]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3548132a5f566c14dd00b4aafcb3f45a844b1b51
[Has the fix been verified in Nightly?]: Can't be completely verified since the change only affects release versions.
[Needs manual test from QE? If yes, steps to reproduce]: No
[List of other uplifts needed for the feature/fix]: https://reviewboard.mozilla.org/r/238202/ (also in this bug)
[Is the change risky?]: I don't believe so
[Why is the change risky/not risky?]: It only contains changes to JSON fields that control whether the telemetry probes are recorded in release versions, as well as flags that link to this bug and add my email as a contact
[String changes made/needed]: None
(Assignee)

Updated

11 months ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 12

11 months ago
Comment on attachment 8969444 [details]
Bug 1454733 - Add opt-out flag, alert email and bug numbers to PWMGR probes;

see above
Attachment #8969444 - Flags: approval-mozilla-beta?
(Assignee)

Comment 13

11 months ago
(copying from IRC for context)

Since the PWMGR probes are all pre-release, we are asking for them to be uplifted this cycle in order to have baseline data on PWMGR usage prior to launching the Lockbox stand-alone iOS app in June. Having earlier data will help us understand if usage of our app (once launched) changes users' interaction with the password manager in the desktop browser. To
10:45 AM be clear: we are not adding any new probes, simply switching existing ones from prerelease to release. Furthermore, since the changes would only affect behavior in release builds, its my understanding that they shouldn't have any effects on nightly or beta.
Comment on attachment 8969445 [details]
Bug 1454733 - Remove PWMGR probes from bug number and alert email whitelists;

Sounds truly low risk from the extra assurances from the telemetry team. 
Let's uplift for beta 16.
Attachment #8969445 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
status-firefox60: --- → affected
Attachment #8969444 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Status: REOPENED → RESOLVED
Last Resolved: 11 months ago11 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.