1804536 - Implement telemetry for weather suggestions

Status: VERIFIED FIXED

(Firefox :: Address Bar, task, P2)

Description

[Drew Willcoxon :adw]

https://mozilla-hub.atlassian.net/browse/SNT-333
https://mozilla-hub.atlassian.net/browse/SNT-348

Comment 1

[Drew Willcoxon :adw]

Attachment: Bug 1804536 - Implement telemetry for weather suggestions.

This adds the following scalars:

• impression_weather
• click_weather
• help_weather
• block_weather

And these histograms:

• FX_URLBAR_MERINO_LATENCY_WEATHER_MS
• FX_URLBAR_MERINO_RESPONSE_WEATHER

The histograms are updated in addition to the existing general Merino latency
and response histograms. I also modified the existing response histogram by
adding a new no_suggestion category so we can tell the difference between a
successful fetch with suggestions and a successful fetch without suggestions.

There's other telemetry in https://mozilla-hub.atlassian.net/browse/SNT-333 that
this doesn't add. I didn't want to do it all here since some of it is very
different. I'll file new bugs as necessary.

Other changes this makes:

• Factor out weather initialization from test_weather.js into MerinoTestUtils so
  it can also be used in the new browser_telemetry_weather.js
• Copy updateTopSites() from the main urlbar head.js to quicksuggest's head.js
• Add some more info() logging to the telemetry helpers in head.js

Comment 2

[Drew Willcoxon :adw]

Attachment: request.md

Data review request for new probes related to the new weather suggestions feature of Firefox Suggest

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:adw, could you have a look please?
If you still have some work to do, you can add an action "Plan Changes" in Phabricator.
For more information, please visit auto_nag documentation.

Comment 4

[Drew Willcoxon :adw]

Waiting on data review

Comment 5

[Elise Richards]

Comment on attachment 9308605 [details]
request.md

I think the following measurements are better represented as Category 1 (technical data) rather than interaction data since they are referring to the latency and responses from merino itself rather than a user interaction:

• FX_URLBAR_MERINO_LATENCY_WEATHER_MS
• FX_URLBAR_MERINO_RESPONSE
• FX_URLBAR_MERINO_RESPONSE_WEATHER

Other than that, I am happy to data-review+ once those are updated 👍

Comment 6

[Drew Willcoxon :adw]

Attachment: request.md v2

Thanks! Here's an updated request.

Comment 7

[Elise Richards]

Comment on attachment 9310972 [details]
request.md v2

Data Review Form

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

   • Yes, through the Histograms.json and Scalars.yaml files, and in the Probe Dictionary and the address bar documentation.

2. Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism) Provide details as to the control mechanism available.

   • Yes, through the normal data-collection opt-out in Firefox settings.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

   • Yes, this a permanent collection and will be monitored by Drew Willcoxon ([e-mail], [e-mail]).

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

   • Category 1 (technical data) and category 2 (interaction data)

5. Is the data collection request for default-on or default-off?

   • Default-on.

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

   • No

7. Is the data collection covered by the existing Firefox privacy notice?

   • Yes

8. Does the data collection use a third-party collection tool? If yes, escalate to legal.

   • No

Result

data-review+

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9b782e3bef55
Implement telemetry for weather suggestions. r=daleharvey

Comment 9

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/9b782e3bef55

Comment 10

[Carmen Fat [:cfat] - Ecosystem QA]

I have verified the following telemetry probes on the latest Firefox Nightly 110.0a1 (Build ID: 20230109093059) on Windows 10 x64, macOS 12.4, and Ubuntu 20.04 x64.

Keyed Scalars:

• Contextual.services.quicksuggest.exposure_weather - When a weather suggestion is triggered
• Contextual.services.quicksuggest.impression_weather - When a weather suggestion is shown, but a different result is clicked
• Contextual.services.quicksuggest.click_weather - When the weather suggestion is clicked
• Contextual.services.quicksuggest.help_weather - When the “Learn more” button from a weather suggestion is clicked
• Contextual.services.quicksuggest.block_weather - When the “Block” option from a weather suggestion is clicked

Histograms:

• FX_URLBAR_MERINO_LATENCY_WEATHER_MS
• FX_URLBAR_MERINO_RESPONSE_WEATHER