1895797 - New Tab Weather Widget: Add Glean Telemetry

Status: VERIFIED FIXED

(Firefox :: New Tab Page, enhancement)

Description

[Maxx Crawford [:maxx]]

Add telemetry items for the following events:

• Clicks on provider links
• Changes default display
• Impressions Moved to 1898275

Comment 1

[Maxx Crawford [:maxx]]

Attachment: Bug 1895797 - Add telemetry to New Tab weather widget r?#home-newtab-reviewers

Comment 2

[Maxx Crawford [:maxx]]

Attachment: bug-1895797-data-review.txt

Sergio - this is my first data review request. Please let me know if you need more information, etc. Thanks!

Comment 3

[Maxx Crawford [:maxx]]

Update: After a conversation with Sergio, and reviewing the new Data Collection docs, this update is a data-classification-low issue and can bypass a formal review.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/89e693330391
Add telemetry to New Tab weather widget r=home-newtab-reviewers,omc-reviewers,nbarrett,aminomancer

Comment 5

[Maxx Crawford [:maxx]]

Attachment: Bug 1895797 - Add error ping to New Tab weather widget when it doesn't load correctly r?#home-newtab-reviewers

Comment 6

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/89e693330391

Comment 7

[Alice Varro [avarro], Ecosystem QA]

I have verified this enhancement using Firefox Nightly 128.0a1, Build ID: 20240520214633 and I was able to trigger events only when the "Switch to simple view/detailed view" option is pressed and the widget is clicked. However, no impression ping was received.
Could you please confirm or not that these are the only events implemented for this feature?
Thanks!

Comment 8

[Scott [:thecount] Downe]

I am unsure how to currently test Glean pings.

Locally what I do is provide an ENV when I build Firefox, which might not be a great solution your work flow.

The other option is to get access to the data, which I also don't know how to do.

We did our testing on our end, but with Glean I don't know how to make this easier.

We might be limited to regression testing from QA unless you have any ideas or experience testing Glean without building Firefox locally.

Comment 9

[Alice Varro [avarro], Ecosystem QA]

We perform our Glean testing by navigating to "about:glean" page, add a tag name in the first field from "About Testing", choose the type of pings we need to see (in this case newtab) and then click the "Apply Settings and submit pings" button.
The pings will be registered in the Glean Debug Viewer page: https://glean-debug-view-dev-237806.firebaseapp.com/

Comment 10

[Maxx Crawford [:maxx]]

Alice - Thank you the QA review. I moved the Impressions event requirement to a follow-up bug

Comment 11

[Maxx Crawford [:maxx]]

Attachment: Bug 1895797 - Add telemetry to New Tab weather widget r=thecount

Comment 12

[Scott [:thecount] Downe]

@alice, oh wow that's way better than the way I was shown.

I think then do you have these set to true? That may be what we're missing.

browser.newtabpage.activity-stream.telemetry

browser.newtabpage.ping.enabled

browser.newtabpage.activity-stream.telemetry.ut.events

Comment 13

[Alice Varro [avarro], Ecosystem QA]

Only "browser.newtabpage.ping.enabled" pref is not set at all on my end, the rest of the two prefs are set as true.

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)

Comment 15

[Scott [:thecount] Downe]

Try setting "browser.newtabpage.ping.enabled" and see if that fixes it. You may have to restart after setting these, not sure.

Comment 16

[Alice Varro [avarro], Ecosystem QA]

(In reply to Scott [:thecount] Downe from comment #15)

Try setting "browser.newtabpage.ping.enabled" and see if that fixes it. You may have to restart after setting these, not sure.

I've set all the prefs to true and all I've managed to trigger is "newtab.weather_change_display" when changing the display mode to simple or detailed and "newtab.weather_open_provider_url" events. The other options from the context menu of the Weather widget do not generate any sort of events and no impression pings were sent to Glean.
You can see the pings I've received in Glean here.

Comment 17

[Scott [:thecount] Downe]

Looking at the code, we only introduced events for those 2 items. So that looks right. Original comment also explicitly specifies we only intended those 2 events in this bug.

I can double check if we intended more events and if they got missed or not, but for the case of this bug, let's just look at the 2 events that landed.

If it turns out we wanted more events, we'll do that in a new bug.

Comment 18

[Alice Varro [avarro], Ecosystem QA]

Thanks for clarifying this! In this case I'll mark this enhancement as verified for Firefox Nightly 128.0a1.

Comment 19

[Scott [:thecount] Downe]

[Tracking Requested - why for this release]: Some telemetry fixes we need for an experiment we want to run in 127.

Because this regressed bug 1896004 and we fixed that in nightly in bug 1898287, we are creating a new phabricator patch for beta, as the previous one that went into nightly would have issues and cause the regression again. The new phabricator patch that's intended for beta should merge to beta clean and create the regression. It is a proper representation of what we intended to land the first time in nightly.

Comment 20

[Phabricator Automation]

Comment on attachment 9403370 [details]
Bug 1895797 - Add telemetry to New Tab weather widget r=thecount

Revision D211296 was moved to bug 1896004. Setting attachment 9403370 [details] to obsolete.

Comment 21

[Scott [:thecount] Downe]

I've tried twice now to create an uplift request for https://phabricator.services.mozilla.com/D211296

Once last night and once this morning. Both times when I hit submit, the dialog closes, I go back to the bug, and the uplift request is not there, no comment, and nothing on the phab attachment.

I am not sure what I am doing wrong or if this is something out of my control.

Comment 22

[Scott [:thecount] Downe]

Attachment: Bug 1895797 - Add telemetry to New Tab weather widget r=thecount

Original Revision: https://phabricator.services.mozilla.com/D211296

Comment 23

[Phabricator Automation]

beta Uplift Approval Request

• User impact if declined: Impacts an experiment in 127, this would delay users getting access to this feature, browser.newtabpage.ping.enabled, and browser.newtabpage.activity-stream.telemetry.ut.events to true.
• Code covered by automated testing: yes
• Fix verified in Nightly: yes
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: 1. Set browser.newtabpage.activity-stream.system.showWeather, browser.newtabpage.activity-stream.telemetry,
• Risk associated with taking this patch: Low
• Explanation of risk level: The changes are adding a new event, new action, and new Glean ping. It's also used exclusivly in a feature that's behind a pref.
• String changes made/needed: None
• Is Android affected?: no

Comment 24

[Phabricator Automation]

beta Uplift Approval Request

• User impact if declined: Impacts an experiment in 127, this would delay users getting access to this feature.
• Code covered by automated testing: yes
• Fix verified in Nightly: yes
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: 1. Set browser.newtabpage.activity-stream.system.showWeather, browser.newtabpage.activity-stream.telemetry, , browser.newtabpage.ping.enabled, and browser.newtabpage.activity-stream.telemetry.ut.events to true.
• Risk associated with taking this patch: Low
• Explanation of risk level: The changes are adding a new event, new action, and new Glean ping. It's also used exclusivly in a feature that's behind a pref.
• String changes made/needed: None
• Is Android affected?: no

Comment 25

[Phabricator Automation]

beta Uplift Approval Request

• User impact if declined: Impacts an experiment in 127, this would delay users getting access to this feature.
• Code covered by automated testing: yes
• Fix verified in Nightly: yes
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: 1. Set browser.newtabpage.activity-stream.system.showWeather, browser.newtabpage.activity-stream.telemetry, , browser.newtabpage.ping.enabled, and browser.newtabpage.activity-stream.telemetry.ut.events to true. 2. Go to about:glean and setup glean to test newtab pings. 3. Load newtab and interact with the weather widget for clicking on "switch to detail view", and clicking on the weather report. Both should fire glean events for "weather_change_display", and "weather_open_provider_url"
• Risk associated with taking this patch: Low
• Explanation of risk level: The changes are adding a new event, new action, and new Glean ping. It's also used exclusivly in a feature that's behind a pref.
• String changes made/needed: None
• Is Android affected?: no

Comment 26

[Scott [:thecount] Downe]

I did the above request in phabricator, and the input boxes where small and caused some issue, so I had to edit it a couple times, which seems to have posted it to bugzilla for each edit. Sorry about that.

Comment 27

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/ebd12b4a5418

Comment 28

[Alice Varro [avarro], Ecosystem QA]

I have verified this enahncement using Firefox Beta 127.0b7 (Build ID: 20240526171419) downloaded from Treeherder and I can confirm that the "newtab.weather_change_display" and "newtab.weather_open_provider_url" events are successfully sent to Glean.