1559136 - Add urlbar event telemetry behind a pref

Status: RESOLVED FIXED

(Firefox :: Address Bar, task, P1)

Description

[Marco Bonardo [:mak]]

Measure the time from the first user input to the execution of an action in the urlbar.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a task, but please change it back in case of error.

Comment 2

[Marco Bonardo [:mak]]

Implementation is blocked on bug 1560585

Comment 4

[Marco Bonardo [:mak]]

Attachment: Bug 1559136 - Add urlbar event telemetry behind a pref. r=adw

Comment 5

[Marco Bonardo [:mak]]

On Monday I'll file the data-review.

Comment 6

[Marco Bonardo [:mak]]

Attachment: data-review-form.md

Ryan, please also check the Events.yaml definitions provided in the patch.

In particular, this will generate events like:
urlbar#engagement#click
urlbar#engagement#enter
urlbar#engagement#drop_go
urlbar#engagement#paste_go
urlbar#abandonment#blur

The value of these events is the type of initial interaction: "typed", "pasted", "dropped", "topsites"

Ancillary data is: elapsed time, number of characters, selection index, selection type (the last 2 are not defined for abandonments)

Afaict, we generate scalars counting the number of these events, even if recording is disabled. Recording can be enabled flipping the browser.urlbar.eventTelemetry.enabled pref (disabled by default).

Comment 7

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Data stewardship programming note: please flag one of the data stewards listed at https://wiki.mozilla.org/Firefox/Data_Collection for data-review; thanks!

Comment 8

[Ryan Harter [:harter]]

Comment on attachment 9079943 [details]
data-review-form.md

As Tim noted, I am not a data steward so I cannot give data review.

Comment 9

[Marco Bonardo [:mak]]

Comment on attachment 9079943 [details]
data-review-form.md

Chutten and Teon are on PTO for a few days yet, we'd like to solve this sooner, trying with Ben.

Comment 10

[Marco Bonardo [:mak]]

Ryan, could you please still check the event covers all of your needs please?

Comment 11

[Ryan Harter [:harter]]

Comment on attachment 9079943 [details]
data-review-form.md

Ben is out as well. Adding Megan.

Comment 12

[Megan McCorquodale]

Comment on attachment 9079943 [details]
data-review-form.md

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way? 
Data will be documented in the probe dictionary. 
2. Is there a control mechanism that allows the user to turn the data collection on and off? 
Yes, can be turned off in preferences. 
3. If the request is for permanent data collection, is there someone who will monitor the data over time?
Yes, harter and the search team.
4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
Category 2 - Interaction Data
5. Is the data collection request for default-on or default-off?
Default on. 
6. Does the instrumentation include the addition of any new identifiers?
No new identifiers. 
7. Is the data collection covered by the existing Firefox privacy notice?
Yes.
8. Does there need to be a check-in in the future to determine whether to renew the data? 
No. 
9. Does the data collection use a third-party collection tool? 
No. 
—
Data review +

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e1231836b044
Add urlbar event telemetry behind a pref. r=adw

Comment 14

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/e1231836b044

Comment 15

[Ryan Harter [:harter]]

Thanks, Mak. Teon's probably the right person to review the telemetry form, but I can take a look as well. I'm having a hard time interpreting the events.yaml file. Can you include some example pings in this bug?

Comment 16

[Marco Bonardo [:mak]]

I think the description I made in comment 6 covers it pretty much, it's one of those events, with a value and an extra object. All the values (also in extra) are strings, per event telemetry requirement.
Tomorrow nightly can be used to try recording events, by setting the pref.
If you have an example of another bug where the kind of ping you want to see was generated, I'm happy to look into doing that.

Comment 17

[Marco Bonardo [:mak]]

I'll also file a bug to add in-tree documentation explaining each value.

Comment 18

[Marco Bonardo [:mak]]

Comment on attachment 9079086 [details]
Bug 1559136 - Add urlbar event telemetry behind a pref. r=adw

Beta/Release Uplift Approval Request

• User impact if declined: No user impact, we need this measurement for experiments that we plan to run from Firefox 69 on.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce: It's not necessary because it's not user-facing
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Event recording is behind a pref, disabled by default. The added methods are no-op when the pref is disabled and they are built to never throw. It will have no impact at all until we enable it for specific experiments population.
• String changes made/needed:

Comment 19

[Ryan VanderMeulen [:RyanVM]]

This needs a rebased patch for Beta.

Comment 20

[Marco Bonardo [:mak]]

Attachment: Beta patch

Comment 21

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9081572 [details] [diff] [review]
Beta patch

Needed to gather Telemetry on experiments slated to run in Fx69. Approved for 69.0b10.

Comment 22

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-beta/rev/f6ce88e03a22

Comment 23

[Marco Bonardo [:mak]]

I'll move the discussion with data-science to a new bug.